SBN Workflow Tools

SBN Workflow Tools

sbnutil github repo

clean_sam_scratch_locations.py
Purpose: Clean dead scratch locations.  Update parameter loc.scratch.

Usage:

clean_sam_scratch_locations.py [options]

Options:

-h|--help             - Print help.
-e|--experiment <exp> - Experiment (default $SAM_EXPERIMENT).
-n|--nfiles <n>       - Number of files to query per iteration (default no limit).
--def <defname>       - Parent definition (optional, default none).
--file <filename>     - File name (optional, default none).
--niter <niter>       - Number of iterations (default 1).
--nolabel             - Check all files with locations, but no tape label.
--invalid_disk <file> - Save files sith invalid persistent disk locations in specified file.
--invalid_tape <file> - Save files sith invalid tape locations in specified file.
dcache_config.py
Purpose: Analysis dCache configuraiton.

Usage:

dcache_config.py [options]

Options:

-h|--help       - Print help.
-e|--experiment - Experiment (default $EXPERIMENT).
--min_depth <n> - Minimum depth to print out (default 3).
--max_deptn <n> - Maximum depth to analyze (default 7).
--md            - Output in markdown format (default plain text).
isartroot.py
Purpose: Test whether file is an artroot file.  With no options, return 
         exit status 0 if file is an artroot file, nonzero otherise.

Usage:

isartroot.py [options] <file>

Options:

-h|--help    - Print help.
-n|--invert  - Invert selection (return status 0 for non-artroot root file).
-a|--anyroot - Return status 0 for any valid root file.
-v|--verbose - Print a human-readable message that matches return status.

Arguments:

<file> - Path of file.
migrate_sam_definitions.py
Purpose: Migrate dataset definitions from source SAM database (SBND or 
         ICARUS) to target SAM database (SBN).

Usage:

migrate_sam_definitions.py [options]

Options:

-h|--help             - Print help.
-e|--experiment <exp> - Experiment (default $SAM_EXPERIMENT).
-d|--definition <def> - Migrate a particular definition (default none).
-n|--ndefinitions<n>  - Number of defiitions migrate (default no limit).
migrate_sam_files.py
Purpose: Migrate files from source SAM database (SBND or ICARUS) to 
         target SAM database (SBN).

Usage:

migrate_sam_files.py [options]

Options:

-h|--help             - Print help.
-e|--experiment <exp> - Experiment (default $SAM_EXPERIMENT).
-n|--nfiles <n>       - Number of files to query per iteration (default no limit).
--def <defname>       - Parent definition (optional, default none).
--file <filename>     - File name (optional, default none).
--niter <niter>       - Number of iterations (default 1).
--invalid <file>      - Save unmigrated files in specified file.
migrate_sam_users.py
Purpose: Migrate users and groups from source SAM database (SBND or
         ICARUS) to target SAM database (SBN).

Usage:

migrate_sam_users.py [options]

Options:

-h|--help             - Print help.
-e|--experiment <exp> - Experiment (default $SAM_EXPERIMENT).
-u|--user <user>      - Check particular user (default all).
sbnpoms_chooser.sh
Purpose: Select artroot file(s), and perform various other "between exe"
         operations.

Usage: sbnpoms_chooser.sh [options]

Options:

-h|--help        - Print help message.
-S <list>        - Specify list file to receive chosen files (default none).
-d <directory>   - Specify directory to search for root files (default ".").
-n <n>           - Number of artroot files to choose (default 1).
--[no]metadata   - [Do not] extract metadata (using sbnpoms_metadata_extractor.py) 
                   for any artroot file in the input directory into a matching 
                   .json file, if the .json file doesn't already exist.
                   Default is to extract metadata.
--delete <list>  - Delete files in the specified list file.
--[no]match      - [Do not] match unpaired non-artroot root files and 
                   unpaired json files.  Rename json file to match root file.
                   Default is to match.
--max_length <n> - Maximum file name length (0=no limit, default 200).
--unique         - Ensure that .root file names are unique (rename). 
sbnpoms_flux_injector.sh
Purpose: Add genie flux-related overrides to fcl file.

Usage: sbnpoms_flux_injector.sh [options]

Options:

-h|-?|--help                - Print help message.
--fcl <fcl>                 - Fcl file to append (default standard output).
--flux_copy_method <method> - Flux copy method (default "IFDH").
--max_flux_file_mb <n>      - Maximum size of flux files to copy (default GENIEGen decides).
sbnpoms_metadata_extractor.py
Purpose: SAM metadata extractor for artroot and non-artroot files.
         Use sam_metadata_dumper to extract internal sam metadata from
         artroot files.  Otherwise, read metadata from associated .json
         file.  Json format metadata written to standard output.

Usage:

sbnpoms_metadata_extractor.py [options] <file>

Arguments:

<file> - Path of file.

Options:

-h|--help - Print help.
-e|--experiment <exp> - Experiment (default $SAM_EXPERIMENT).
sbnpoms_metadata_injector.sh
Purpose: Append SAM metadata overrides to a fcl file.

Usage: sbnpoms_metadata_injector.sh [options]

General options:

-h|-?|--help                  - Print help message.
--inputfclname <fcl file>     - Fcl file to append to.

Options for overriding SAM built-in metadata (service FileCatalogMetadata).

--mdappfamily <family>        - Application family.
--mdappversion <version>      - Appliction version.
--mdfiletype <file_type>      - File type.
--mdruntype <run_type>        - Run type.
--mdgroupname <group>         - Group.

Options for overriding experiment-specific metadata.

--mdfclname <fcl file>        - Fcl file name to store in metadata.
--mdprojectname <project>     - Project name.
--mdprojectstage <stage>      - Project stage.
--mdprojectversion <version>  - Project version.
--mdprojectsoftware <product> - Top level ups product.
--mdproductionname <campaign> - Campaign name.
--mdproductiontype <type>     - Campaign type.

Options for non-artroot files.

--tfilemdjsonname <json name> - Name of TFile json file.
--cafname         <caf name>  - Name of caf file.
sbnpoms_runnumber_injector.sh
Purpose: Append run and subrun overrides to fcl file.
         Subrun follows \$PROCESS+1, and wraps by incrementing
         the run number when the subrun exceeds the maximum.

Usage: sbnpoms_runnumber_injector.sh [options]

Options:

-h|-?|--help          - Print help message.
--fcl <fcl>           - Fcl file to append (default standard output).
--subruns_per_run <n> - Number of subruns per run (default 100).
--process <process>   - Specify process number (default \$PROCESS).
--run <run>           - Specify base run number (default 1).
sbnpoms_hepevt_extractor.sh
-h|-?|--help          - Print help message.
-i|--input            - Specify input hepevt text file (no default).
-o|--output           - Specify output hepevt text file
                        (default HEPevents.txt in current directory).
-n <nev>              - Number of events to extract (default 20).
--nskip <nev>         - Number of events to skip (default 0).
--nskip_process <nev> - Number of events to skip per process (default same as -n).
--process <proc>      - Override $PROCESS environment variable for the
                        purpose of calculating skip events.
sbnpoms_wrapperfcl_maker.sh
Purpose: Make an empty wrapper fcl file.

Usage: sbnpoms_wrapperfcl_maker.sh [options]

Options:

-h|-?|--help        - Print help message.
--fclname <fcl>     - Wrapped fcl file.
--wrappername <fcl> - Wrapper fcl file.
sbnd/sbndpoms_caf_concatenation.sh
Purpose: Produces concatenated caf files from a dataset of standard cafs.
         Also provides the option to flatten the concatenated file.

Usage: sbndpoms_caf_concatenation.sh --defname <SAMDEF> [options]

Options:

-h|-?|--help         - Print help message.
--workdir <dir>      - Top level directory for producing the files.
--outdir <dir>       - Top level directory for storing the files.
--size <maxsize>     - Specify the maximum size of one of the files.
--flatten            - Created flattened caf files too.
sbnd/sbndpoms_genfclwithrunnumber_maker.sh
Purpose: Wrapper script to create a set of initial fcl files.
         Makes use of the sbnpoms_metadata_injector.sh script.

Usage: sbndpoms_genfclwithrunnumber_maker.sh [options]

Options:

-h|-?|--help         - Print help message.
--fcl <fcl>          - The gen fcl file for this campaign.
--outdir <dir>       - Top level directory for storing the files.
--samdeclare         - Declare produced fcls to SAM.
--nfiles <n>         - Number of files (i.e. subruns) to produce.
--workdir <dir>      - Top level directory for producing the files.
--mdprojver <ver>    - Code version being used for campaign.
--mdprojname <name>  - Name of production (i.e. MCPXXXXA).
--mdprojtype <type>  - Type of production (i.e. official).
--mdstagename <name> - Name of stage (i.e. gen).
-v|--verbose         - Increases verbosity of output.