1. LArSoft Data Products : Larsoft website Larsoft github RecoBase
2. LArSoft Versions: LArSoft release list
3. DUNE Computing tutorial (January 2023) : indico.fnal.gov/event/57711
Items discussed during the tutorial :4. DUNE Computing Wiki Pages: dunescience.org Links to Computing basic (ex. Githib) and DUNE-Specific tutorials are available.
5. Cold-Box data reconstruction
Summary of the reconstruction status: list of reconstructed runs, code version, reconstruction results availabity... wiki.dunescience.org/wiki/ProtoDUNE_Vertical_Drift_operation_status —> Analysis ntuple are also available at CC-IN2P3 here : /sps/lbno/coldbox
More details on the organization of the reconstruction campaigns :
The development of the reconstruction software, the validation and analysis of reconstruction results have been presented during several Cold-Box and protoDUNE-VD simulation and reconstruction meetings6. DUNE Working groups:
Hits: 2D representation of charge deposited in the TDC/wire plane
Hits are assumed to be made from deconvoluted, unipolar, calibrated signals. They identify a charge deposit in a specific location and time. the location is absolute and unique in the detector, while the time is relative to the start of sampling (tick time).
Cluster : Set of hits with a 2D structure. Set of reconstructed hits supposed to originate from the same physical event. A cluster lies in a single plane (of a single TPC).
SpacePoints : 3D objects that contain pointers to Hits from multiple wireplanes that have been identified as matching
TrackTrajectory: contains a trajectory in 6D space representing the path walked by a particle. A trajectory point is made of a 3D position component (measured in centimeters) and a momentum component (measured in GeV/c)
Track: object including TrackTrajectory plus fitted track (chi2, number of dof, particle ID, vertices). A `recob::Track` consists of a `recob::TrackTrajectory`, plus additional members relevant for a "fitted" track
void GetHitTrk(gallery::Event const& ev, InputTag const& hit_tag,
InputTag const& track_tag, vector<int> &hit_id, vector<int> &hit_ch){
auto const hitlist = ev.getValidHandle<vector<recob::Hit>>(hit_tag);
art::FindManyP<recob::Track> fmp(hitlist,ev,track_tag);
hit_id.resize(hitlist->size(),0);
hit_ch.resize(hitlist->size(),0);
for (size_t i = 0, sz = hitlist->size(); i != sz; ++i) {
if (fmp.isValid()){
hit_ch[i] = hitlist->at(i).Channel();
if (fmp.at(i).size()!=0)
hit_id[i] = fmp.at(i)[0]->ID();
else
hit_id[i] = -1;
}
}
}
art::FindManyP<recob::Track> fmp(Hits ,ev ,fTrackModuleLabel);
Producer_adcprep_notool -> channel mapping, ADC calibration, coherent noise removal (raw::RawDigit -> raw::Wire )
RandomNumberSaver -> store the random number generator seed within the event in order to be able to retrieve it later
Wirecell -> deconvolution of the electronic response, bipolar to unipolar signal (raw::Wire -> recob::Wire)
Gaushit -> GausHitfinder -> the algorithm walks along the wire and looks for pulses above threshold and then attempts to fit n-gaussians to these pulses where n is set by the number of peaks found in the pulse (recob::Wire -> recob::Hit)
Cette partie liste les différents tutos déjà produits
-------------------------------
- Launch job tutorial -
-T.Houdy adpated from L.Zambelli-
- (23/11/2022) -
-------------------------------
1. Login at CERN
2. Create 3 files :
script_condor.sub, script_condor.sh, runs.txt
3. Run the command:
$ condor_submit script_condor.sub
--------------------------------
with script_condor.sub :
-------------------------------------------------------------------------------
# HTCondor queues
# espresso = 20 minutes, microcentury = 1 hour, longlunch = 2 hours, workday = 8 hours, tomorrow = 1 day, testmatch = 3 days, nextweek = 1 week
# notify_user = prenom.nom@cern.ch
# Never, Error, Complete, Always
notification = Error
universe = vanilla
executable = /your/local/path/script_condor.sh
output = /your/local/path/reco_$(ProcId).out
error = /your/local/path/reco_$(ProcId).log
log = /your/local/path/reco_$(ProcId).err
## Force HTCondor not to use shared filesystem
#priority = 20
#request_memory = 1000 MB
##request_disk = 10 GB
+JobFlavour = "workday"
#+MaxRuntime = 90000
#+RequestCpus = 1
queue arguments from runs.txt
-------------------------------------------------------------------------------------
--------------------------------
with script_condor.sh :
-------------------------------------------------------------------------------------
#!/bin/bash
source ~/.lar_profile
run=$1
lar -c your_fickle.fcl (your_input_file.root) -o output_file_$run.root
-------------------------------------------------------------------------------------
--------------------------------
with runs.txt:
-------------------------------------------------------------------------------------
1
2
3
4
5
6
--------------------------------------------------------------------------------------
#!/bin/bash
#-------------------------------
#- SubmitFNALjobs -
#- Y.Kermaidic -
#- (23/03/2023) -
#-------------------------------
SCRIPT_PATH=/dune/app/users/username/scripts
TOP_RUN=17
RECO_PATH=/pnfs/dune/tape_backed/dunepro/path/to/your/reco/data/00/00/${TOP_RUN}
OUTPUT_PATH=/pnfs/dune/resilient/users/username/your/output/path
RUNS=$(ls ${RECO_PATH});
source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
setup jobsub_client
for RUN in ${RUNS[@]};
do
# Assumes that you have the list of files you want to process in OUTPUT_PATH, and called "${TOP_RUN}${RUN}_reco.list"
RUNLISTJOB="${TOP_RUN}${RUN}_reco.list"
RUNLIST="${OUTPUT_PATH}/${RUNLISTJOB}"
OUTPUT_FILE="output-filename_${TOP_RUN}${RUN}.root"
echo " "
echo "SubmitFNALjobs.sh : "
echo " - RUNLIST: ${RUNLIST} "
echo " - OUTPUT_FILE: ${OUTPUT_FILE}"
echo " "
# Interactive start for initial check of the script
#bash ${SCRIPT_PATH}/RunFNALscript.sh ${RUNLISTJOB} ${OUTPUT_FILE}
# Job submission command
jobsub_submit -N 1 -M --OS=SL7 --group=dune --expected-lifetime=3h --resource-provides=usage_model=DEDICATED,OPPORTUNISTIC file://${SCRIPT_PATH}/RunFNALscript.sh ${RUNLISTJOB} ${OUTPUT_FILE}
done;
#!/bin/bash
#-------------------------------
#- Reconstructed files listing -
#- Y.Kermaidic -
#- (23/03/2023) -
#-------------------------------
source ~/.larsoft_profile
# you may need to run 'kx509' first for validating your certificate to access the samweb database
SCRIPT_DIR=$( pwd )
PROD_KEY=2023-03
TOP_RUN=17
INPUT_DIR=/pnfs/dune/tape_backed/dunepro/vd-coldbox-top/full-reconstructed/2023/detector/test/VD_coldbox_CRP2_CRP3_2022/00/00/${TOP_RUN}
OUTPUT_DIR=/your/path/where/to/store/the/xrootd/list/of/files
echo "INPUT_DIR: ${INPUT_DIR}"
cd ${INPUT_DIR}
RUNS=$( ls )
for RUN in ${RUNS[@]}
do
echo "RUN: ${RUN}"
cd ${RUN}
FILES=$( ls ${TOP_RUN}${RUN}*_${PROD_KEY}*.root )
for FILE in ${FILES[@]}
do
FULL_PATH=$( samweb get-file-access-url --schema=xroot ${FILE} )
echo "FILE: ${FILE} -> ${FULL_PATH}"
echo "${FULL_PATH}" >> ${OUTPUT_DIR}/${TOP_RUN}${RUN}.list
done
cd -
done
cd ${SCRIPT_DIR}
-------------------------------
- Compile local LArSoft tutorial -
- T.Houdy -
- (3/02/2023) -
-------------------------------
-------------------------------------------------------------------------------
1. Login at CCIN2P3/CERN/FNAL
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
For your first time:
-------------------------------------------------------------------------------
2. Choose the place you want to settle your local LArSoft software
-------------------------------------------------------------------------------
$ mkdir dune_workdir
$ cd dune_workdir
$ source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
$ mrb newDev -v v09_63_00d01 -q prof:e20
$ source localProducts_larsoft_v09_63_00d01_prof_e20/setup
$ echo $MRB_PROJECT
return: "
MRB_Project =
...
"
-------------------------------------------------------------------------------
3. To get the source code
-------------------------------------------------------------------------------
$ cd $MRB_SOURCE
$ mrb g dunesw
$ cd dunesw
$ git checkout develop -b local_tibo_version
-------------------------------------------------------------------------------
4. Download only what you want to modify, the rest will be herited from the corresponding version LArSoft
-------------------------------------------------------------------------------
--------------- To download DUNE/dunereco--------------------------------------
$ cd $MRB_SOURCE
$ mrb g dunereco
$ cd dunereco
$ git checkout develop -b local_tibo_reco
-------To download DUNE/dunesim
$ cd $MRB_SOURCE
$ mrb g dunesim
$ cd dunesim
$ git checkout develop -b local_tibo_sim
-------------------------------------------------------------------------------
5. Check compatibility between versions of source codes
$ cd $MRB_SOURCE
$ cd dunesw
$ mrbsetenv
-------------------------------------------------------------------------------
6. If no error messages, start building
$ cd $MRB_BUILDDIR
$ mrb install -j3
-------If installation is successful
$ mrbslp
$ cd ..
$ lar
--> You can now use your own version of LArSoft using the usual lar command anywhere.
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
-------Later to use your local version
--- To get the DUNE softwares like mrb!
$ source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
--- To load your specific local LArSoft version
$ source dune_workdir/localProducts_larsoft_v09_56_00d00_prof_e20/setup
--- setup the products installed in the localProduct folder (including a specific LArSoft version)
$ mrbsetenv
--- setup the development environment
$ mrbslp
--> You can now use your own version of LArSoft using the usual lar command anywhere.
-----------------------------------------------------------------------------------
-----------------------------------------------------------------------------------
To compile your local version after modifications
$ source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
$ source $YOUR_MODULE/localProducts_larsoft_v09_65_03d00_prof_e20/setup
$ mrbsetenv
$ cd $MRB_BUILDDIR
$ mrb install -j8
$ mrbslp
$ cd /dune/app/users/name/my_analysis/
$ lar
-------------------------------
- LArsoft simulation tutorial -
- Y.Kermaidic -
- (08/02/2022) -
-------------------------------
1. Login (CERN / FNAL):
-----------------------
1. kinit -f username@CERN.CH
2. ssh username@lxplus.cern.ch
3. kinit -a -r7d username@FNAL.GOV
4. ssh -K username@dunegpvm04.fnal.gov
2. Setup environment:
---------------------
1. source .larsoft_profile (see typical content below - from Thomas Kosc (07/02/2022) - dunetpc(Redmine) became dunesw(Github) on 12/2021)
cd /dune/data/users/username
source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
export DUNEVERSION=v09_42_02_00
export DUNEQUALIFIER="e20:prof"
setup dunesw $DUNEVERSION -q $DUNEQUALIFIER
2. export FHICL_CB_CONFIG=/cvmfs/dune.opensciencegrid.org/products/dune/dunesw/v09_42_02_00/fcl
3. Run the production:
----------------------
lar -c $FHICL_CB_CONFIG/vdcb_tde_single_muminus_5GeV_fixedstart.fcl
lar -c $FHICL_CB_CONFIG/vdcb_tde_g4.fcl prod_muminus_5.0GeV_fwdgoing_fixedstart_vdcb_tde_gen.root
lar -c $FHICL_CB_CONFIG/vdcb_tde_detsim.fcl prod_muminus_5.0GeV_fwdgoing_fixedstart_vdcb_tde_gen_g4.root
# These commands will generate the following files:
# - prod_muminus_5.0GeV_fwdgoing_fixedstart_vdcb_tde_gen.root
# - prod_muminus_5.0GeV_fwdgoing_fixedstart_vdcb_tde_gen_g4.root
# - prod_muminus_5.0GeV_fwdgoing_fixedstart_vdcb_tde_gen_g4_detsim.root
# - prod_muminus_5.0GeV_fwdgoing_fixedstart_vdcb_tde_gen_g4_detsim_hist.root
4. Event display:
----------------
lar -c $FHICL_CB_CONFIG/evd_vdcb_tde.fcl prod_muminus_5.0GeV_fwdgoing_fixedstart_vdcb_tde_gen_g4_detsim.root
-----------------------------
- LArsoft dataprep tutorial -
- Y.Kermaidic -
- (07/12/2021) -
- update: 09/02/2022 -
-----------------------------
1. Login (CERN / FNAL):
-----------------------
1. kinit -f username@CERN.CH
2. ssh -Y username@lxplus.cern.ch
3. kinit -a -r7d username@FNAL.GOV
4. ssh -K -Y username@dunegpvm04.fnal.gov
2. Setup environment:
---------------------
1. source .larsoft_profile (see typical content below - from Elisabetta slides (06/12/2021))
cd /dune/data/users/username/dunetpc/larsoft
source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
export DUNEVERSION=v09_42_02_00
export DUNEQUALIFIER="e20:prof"
setup dunesw $DUNEVERSION -q $DUNEQUALIFIER
2. export FHICL_CB_CONFIG=/cvmfs/dune.opensciencegrid.org/products/dune/dunesw/v09_42_02_00/job
3. export RAW_CB_DATA=/pnfs/dune/tape_backed/dunepro/vd-coldbox-top/raw/2021/detector/test/None
3. Run the dataprep:
--------------------
lar -c $FHICL_CB_CONFIG/vdcoldbox_raw_tdedataprep.fcl $RAW_CB_DATA/00/00/04/45/445_50_cb.test
(or lar -c myfhiclfile.fcl $RAW_CB_DATA/00/00/04/45/445_50_cb.test)
4. Plot example:
----------------
root -l 445_50_cb_vdcbtde_dataprep.root
Events->Draw("raw::RawDigits_tpcrawdecoder_daq_VDTDEDataPrep.obj.fPedestal:raw::RawDigits_tpcrawdecoder_daq_VDTDEDataPrep.obj.fChannel")
Events->Draw("raw::RawDigits_tpcrawdecoder_daq_VDTDEDataPrep.obj.fSigma:raw::RawDigits_tpcrawdecoder_daq_VDTDEDataPrep.obj.fChannel")
Events->Draw("raw::RawDigits_tpcrawdecoder_daq_VDTDEDataPrep.obj.fSigma:raw::RawDigits_tpcrawdecoder_daq_VDTDEDataPrep.obj.fChannel", "", "colz")
-------------------------------
- Setup VNC session tutorial -
- Y.Kermaidic -
- (09/02/2022) -
-------------------------------
# Aim to speed-up a lot the event display of LArSoft or LARDON
0. Setup your SSH configuration:
--------------------------------
0. Adapted from https://cdcvs.fnal.gov/redmine/projects/sbndcode/wiki/Viewing_events_remotely_with_VNC
1. On your local machine, put these lines into your ~/.ssh/config - change XX by a 2 digits number (01, ..., 99)
Host dune04
HostName dunegpvm04.fnal.gov
User your_username
ForwardAgent yes
ForwardX11 yes
ForwardX11Trusted yes
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
LocalForward 5901 localhost:59XX
2. On your local machine, make sure your /etc/krb5.conf file is properly set:
Look at https://authentication.fnal.gov/krb5conf/
Add the relevant configuration file to your /etc/krb5.conf
Try to log via:
kinit -a -r7d your_username@FNAL.GOV
-> you should be asked for your Kerberos password
ssh dune04
-> you should be logged in without being asked for any password
2. On your remote machine, put these lines into your ~/.bash_profile
#VNC stuff
VNCNUM=XX #CHANGE THIS NUMBER TO WHATEVER VNC SERVER NUMBER YOU PICKED
if [[ `hostname` == *"gpvm"* ]] #only start VNC servers on the gpvms (i.e. not on the build machines)
then
export DISPLAY=localhost:$VNCNUM #Export the display to point to the VNC server
if [ `lsof -i -P -n | grep $(expr 5900 + ${VNCNUM}) | wc -l` -eq 0 -o `lsof -i -P -n | grep $(expr 6000 + ${VNCNUM}) | wc -l` -eq 0 ]
then
echo "vncserver :$VNCNUM not running. Starting now...."
vncserver :$VNCNUM -localhost -bs #Check if the VNC server is running and start it if not (-localhost mandatory!)
else
echo "vncserver :$VNCNUM already running (hopefully owned by you). Not attempting to start the vncserver..."
fi
fi
2. Log in on your remote machine, start a VNC session and a test display (like ROOT TBrowser):
----------------------------------------------------------------------------------------------
kinit -a -r7d your_username@FNAL.GOV
ssh dune04
vncserver :XX -localhost -bs
-> setup a custom/dedicated password "MYPWD"
source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
setup dunesw v09_42_02_00-q e20:prof
root
new TBrowser
3. On your local machine, open the VNC window:
----------------------------------------------
open vnc://localhost:5901
-> Enter your custom password "MYPWD"
-> You should see a Scientific Linux window opening