Development of a pipelined data acquisition system for µSR experiments at J-PARC Soh Yamagata Suzuki a,∗ , Manobu Tanaka a,b , Kazuya Tauchi a , Yoshiji Yasu a , Soshi Takeshita a , Akihiro Koda a,b , Masatoshi Hiraishi b , Masanori Miyazaki a,b , Kohki Satoh b , Ryosuke Kadono a,b , Katsuhiko Ishida c , Dai Tomono c and Teiichiro Matsuzaki c a High Energy Accelerator Research Organization, Tsukuba, Ibaraki 305-0801, Japan Graduate University for Advanced Studies (Sokendai), Shonan Village, Hayama, Kanagawa 240-193, Japan c Advanced Meson Science Laboratory, The Institute of Physical and Chemical Research (RIKEN), Wako, Saitama 351-0198, Japan b The
Abstract We have developed a pipelined data acquisition (DAQ) system for µSR experiments at J-PARC. From this September, the J-PARC accelerator starts the beam delivery to the muon sources and the first µSR experiment will be in operation. When fully operable, the muon beam intensity delivered at the Muon Experiment Facility would be two or three orders of magnitude greater than that of the Muon Science Laboratory (KEK-MSL). This condition forces us to replace the detector and the DAQ system and the new DAQ system must fit the framework for the run automation at J-PARC. We have developed new DAQ system for that purpose, this will be used in the first experiment and ready to coming new µSR detector for the full operation of the accelerator. Key words: µSR method, DAQ
1. Introduction The High Energy Accelerator Research Organization (KEK) and the Japan Atomic Energy Agency (JAEA) have cooperatively started the Japan Proton Accelerator Research Complex (JPARC). The J-PARC will be used for researches of particle physics, nuclear physics, material science, life science and nuclear technology. The accelerator already started delivering a proton beam to neutron sources and will start to muon sources from this autumn [1]. The expected intensity of the muon beam is two or three orders of magnitude higher than that of KEK Muon Science Laboratory ∗ el. +81-29-864-5481, email:
[email protected] Preprint submitted to Elsevier
(KEK-MSL). This intensive beam will brig high multiplicity of the detector and µSR experiments in J-PARC will encounter a new problem that did not exist in KEK-MSL. 2. µSR experiments in J-PARC From autumn 2008, J-PARC will begin beam delivery to the MUon Science Establishment (MUSE). When the accelerator provides a proton beam in the full operation, the intensity of the muon beam in MUSE will be two or three orders higher than that of KEK-MSL. This high intensity brings the extreme high multiplicity in the detector and we have to replace both of the detector and the data acquisition (DAQ) system. 22 July 2008
S
The spectrometer of µSR experiments consists of segmented scintillators to detect positrons from the µ+ decay. Using pulsed muon beam, the spectrometer must observe many µ+ decays from one bunch. The DAQ system uses a Time-to-Digital Converter (TDC) to measure the time spectrum of the decay events, and this TDC must equip the multi-hit functionality. The segmentation number of the spectrometer in the KEK-MSL is 64 and the DAQ system used CAMAC-TDC, LeCroy 3377 which has a hit buffer of up to 16 hits for each channel. The multiplicity was about 5 hits per pulse. The double pulse resolution of the DAQ system is about 20 ns which is limited by the CAMAC discriminator. The precise measurement of the time spectrum requires the fine double pulse resolution and the deep buffer to hold multi-hit in the TDC. If either of them is insufficient, several hits will be missed and the observed time spectrum will be distorted. A more intensive beam makes higher multiplicity and make it harder to prevent the distortion. If we rebuild the spectrometer with more finer segmentation, the multiplicity becomes lower and the distortion problem will be solved. But if we solve the distortion only by the finer segmentation, the segmentation number will be 64000 and almost impossible. On the other hand, if we solve the problem only by the DAQ system replacement, it must equip the buffer for 5000 hits per channel and 20 ps double pulse resolution. This is also impossible. Therefore our upgrade plan of the spectrometer is that, the segmentation number is up to 2000, the double pulse resolution and the time resolution are better than 1 ns, and the sufficient buffer depth in the TDC. The expected multiplicity is about 160 hits per channel under the 2000 segmented spectrometer.
i
g
n
a
l
s
f
r
o
m
t
h
e
d
e
t
e
c
t
o
r
D
i
g
i
t
i
z
e
r
C
M
D
o
i
g
d
i
u
t
i
l
z
e
o
d
u
U
r
T
M
P
e
l
r
i
g
g
e
r
e E
R
B
u
n
c
h
t
i
m
i
n
e
c
e
i
v
e
t
h
e
r
n
e
t
r
g D
i
g
i
g
M
D
i
o
M
i
t
i
d
i
o
t
d
z
u
e
l
z
u
r
e
e
l
r
e
B
a
s
e
B
o
a
r
d
Fig. 1. copper components
digitizer cards and send to another computer via a network. The form factor of the base board is a 9U VME, but the data transmission does not use the VME bus but the network. The base board equips a FastEthernet for the data transmission. The deadtime of the COPPER system is defined as the data transmission time from pipeline on the digitizer card to the buffer on the base board that is 23 ns per words. The first COPPER system is installed in the DAQ system of the Belle experiment at KEK and already the COPPER-TDC is developed for that is available. But the buffer depth of the TDC module is 256 hits and this hit buffer is shared by 24 channels. This depth maybe insufficient for our requirement and we newly developed the TDC mezzanine card that has the sufficient buffer depth. 4. Experiment 4.1. COPPER applicability In the first step, we tested the applicability of the COPPER system to the µSR experiment. At 2006 winter, we installed the COPPER system with existing TDC modules in the readout system of the detector Advanced Riken General purpose mUsr Spectrometer (ARGUS) [4] at the RIKENRAL Muon Facility [5] of the Rutherford Appleton Laboratory in the UK. At present, this facility provides the strongest pulsed µ beam in the world. The DAQ system for ARGUS and KEK-MSL are similar. The existing TDC module was originally developed for the Belle experiment. The LSB is 780 ps, the time resolution is 0.54 LSB and the buffer depth is 256 hits for 24 channels [3]. This performance but the buffer depth is sufficient for the µSR experiments. Using this COPPER system, we measured the time spectrum and the multiplicity of µ-e decay with
3. COPPER system As there is no commercially available TDC that has the buffer for 160 hits per channel, we decided to supersede the CAMAC system using the “COmmon Platform for Pipelined Electronics Readout (COPPER)”, which was designed by KEK [2]. Fig.1 shows the components of the COPPER system. A COPPER board consists of a 9U size base board, four digitizer mezzanine cards, a trigger receiver and one processor module that is standardized as a PrPMC (IEEE P1368 VITA32). The Linux operating system runs on this processor module, reads data from 2
h
1
0.8
# of events
Entries
5
10
104
1.330819e+07
Mean
1.975
RMS
2.402
3
0
102
-0.2
3
10
htemp
-0.4
102
Entries
10 1
# of events
Mean
-0.6
0
5
10
15
20
25
Mean y RMS
-0.8
30
RMS y
Multiplicity in single channel
-1 0
h Entries Mean RMS
3
10
0.6728
χ / ndf 119.7 / 2 Constant 3816 ± 54.4 Mean 0.2292 ± 0.0084 Sigma 0.6515 ± 0.0043
0.2
h
6
6354 -0.001259
2
10
0.4
10
RMS
0.6
7
10
Entries Mean
# of hits
Difference from the expected value
the silver plate target under the 500 G magnet field. In this test, the collimator is fully opened to get the maximum multiplicity. The maximum occupancy of the hit buffer is about 150 hits per bunch (Fig.2).
78687 96.93 10.15
6343 3.382e+04 -0.0009459 1.831e+04 0.6722
10000 20000 30000 40000 50000 60000 Time since the start signal
10
-5
-4
-3
-2 -1 0 1 2 3 4 5 Difference from the expected value
102
Fig. 3. Linearity of thew new COPPER-TDC. The left graph shows the difference between the time measured by the TDC and the actual signal time from a TDC tester. There was no significant tendency. The right histogram shows the error distribution in the overall range of the TDC. The RMS was 0.65ns.
10
1
0
20
40
60
80
100 120 140 160 Occupancy of hit buffer of AMT-3
Fig. 2. The multiplicity in a single channel and the occupancy of the hit buffer that is shared by 24 input channels. The depth of the hit buffer is 256 and no hits were lost in this test experiment.
trum from the Ag plate in 500G magnetic field. In this test, The beam intensity was three-fold higher than that of 2006 winter and the maximum multiplicity reached 34 hits per channel. But the buffer depth of the new TDC, 1024 hits per channel is enough for that condition.
4.2. New TDC module development The µ beam magnitude of the RIKEN-RAL is 50 times smaller than that of the full operation of the J-PARC and the buffer depth of the existing TDC is almost insufficient for our requirement. In the case of other commercial TDCs such as VME V1290 by CAEN, it uses HPTDC developed by CERN which hit buffer is shared by 16 ∼ 32 channels and the depth is 256 or 512 hits and insufficient. Therefore we have developed new TDC module that has sufficient buffer size, 160 hits per channel at least. Our new TDC module equips one Xilinx SPARTAN XC35400 which contains the TDC logic and the interface logic to the COPPER base board. The specifications of this TDC are following: 16 input channels per module, LSB is 1ns, time resolution is 0.65ns and the depth of the hit buffer is 1024 hits for every channel. When all channels receives 160 hits, this TDC will issue about 60 µsec deadtime. This is negligibly short in the case of µSR experiments the J-PARC. The linearity of this TDC is shown in Fig.3.
h0 Entries 8191306 Mean 6748 RMS 2007
# of hits
time {ch > 31}
104
103
102 6000
8000
10000 12000 14000 Time since the kicker pulse (ns)
Fig. 4. Observed time spectrum using the new TDC with the ARGUS. The collimator is fully opened, but there is no distortion.
5. Integration with the slow control system 5.1. DAQ middleware
4.3. New TDC applicability
Experiments in the J-PARC MLF must be controlled from the “Working Desktop” [6] that is the user interface framework to control the DAQ system, the analyzer system and the environment such as the
We tested this new COPPER-TDC with ARGUS in 2007 and 2008. Fig.4 is the observed time spec3
temperature, the pressure and the beam collimator. The cooperation of the Working Desktop is essential to the automation of the run control. Most of the components are already implemented and used in the neutron experiments in the J-PARC. The DAQ middleware [7] is used for the whole operation of the DAQ system. An unit of this middleware consists of Gatherer, Dispatcher, Logger and Monitor. The Working Desktop can handles multiple units of these middleware via DAQ operators connected to each unit. The performance of the DAQ middleware is confirmed to up to 20MB/s per unit. If the spectrometer requires more higher throughput, multiple units are necessary. Working Desktop
HTTP
XML/HTTP XML/HTTP DAQ operator
Environment operator D
Detector Detector Detector Detector
A
Q
m
i
d
d
l
e
w
a
r
If NA is larger than NB but NA misses some hits, the Asym is smaller than the expected. To prevent this problem, we have to exclude events that contain missing hits. To eliminate such events, an event data must contain all hits from all channels. But the original DAQ middleware works as following: collect data that contains multiple events from the first frontend, send it to the Dispatcher, collect data from the second frontend, send it to the Dispatcher and so on. The number of events in the single data transmission is not constant. We have to rewrite them to collect process the data in event-by-event. The asymmetry histogram is calculated from two group histograms. As the direction of the magnetic field in µSR experiments in J-PARC is various and the monitor must be able to change the grouping dynamically. We prepared the channel selector using the ROOT toolkit (Fig.6). When the Monitor receives the data from the Dispatcher, it has and updates histograms for every channel after excluding events that miss several hits. The channel selector continuously displays the sum of histograms of chosen channels. The displayed histogram is always continuously updated.
XML/HTTP Analyze opertor
CORBA CORBA CORBA CORBA
HTTP
e
Gatherer
Dispatcher
S
t
o
r
a
g
e
s
Monitor Logger
Fig. 5. Components related to the DAQ system. An unit of the DAQ middleware collects data from multiple inputs and send it to the storage. The user controls the run via the working desktop and confirm the status of the run from the output of the monitor. The working desktop can manage multiple units of the DAQ middleware.
5.2. Modification for µSR experiments
Fig. 6. Screenshot of the dynamic channel selector. User can choose channels to be treated as a group. The summed histogram of the channel group is always displayed and continuously during the run.
The DAQ middleware is originally developed for neutron experiments in the J-PARC and lacks an essential function for µSR, that is the way to monitor the correlation of different frontends. µSR experiments must measure the asymmetry of time spectrums from µ-e decays. To minimize the distortion of the time spectrum, all events contains missing hits must be excluded. When NA and NB are total number of hits from channels grouped A, B respectively. the amplitude of the asymmetry is defined as Asym =
6. Summary From September 2008, MUSE will be in operation and we have developed a pipelined DAQ system for µSR experiments for that. At the beginning of MUSE, the beam intensity is not so high and we will reuse the spectrometer which was used in the KEK-MSL. Our first spectrometer consists of 256 scintillators and the first DAQ system for this spec-
NA − NB NA + NB 4
trometer consists of uses two COPPER-TDCs and one DAQ middleware unit. References [1] K. Shimomura, et al., Nucl. Phys. Proc. Suppl. 155, 343 (2006) [2] T.Higuchi, et al., IEEE Trans. Nucl. Sci. 52, 1912 (2005) [3] S. Y. Suzuki, et al., “Performance of the AMT-3 based TDC system at Belle” in Nuclear Science Symposium, 2006. Conference Record, 2006, vol 1, pp. 128-131. [4] R. Kadono, et al., RIKEN Accelerator Progres Report 29, 1996, p. 196; RIKEN-RAL Muon Facility Report 1, 1997, p. 27 [5] T. Matsuzaki, et al., Nucl. Instr. and Meth. A 465 (2001) 365 [6] T. Nakatani et al., Day-1 status of the software framework for experimental instruments in J-PARC/LF, presented at International Symposium on Pulsed Neutron and Muon Sciences (IPS08), March 2008, Mito, Japan. [7] Y. Yasu et al., A Data Acquisition Middleware, Conference Record / 15th IEEE NPSS RealTime Conference, Fermilab, Batavia IL, 60510, April 29 - May 4, 2007
5
