All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Public Member Functions | Public Attributes | List of all members
python.extractor_dict.expMetaData Class Reference
Inheritance diagram for python.extractor_dict.expMetaData:
python.extractor_dict.MetaData

Public Member Functions

def __init__
 
def translateKey
 
def md_gen
 
def getmetadata
 
- Public Member Functions inherited from python.extractor_dict.MetaData
def __init__
 
def extract_metadata_to_pipe
 
def get_job
 

Public Attributes

 expname
 
 translateKeyf
 
 md
 
- Public Attributes inherited from python.extractor_dict.MetaData
 inputfile
 

Additional Inherited Members

- Static Public Member Functions inherited from python.extractor_dict.MetaData
def wait_for_subprocess
 
def mdart_gen
 
def md_handle_application
 

Detailed Description

Class to hold/interpret experiment-specific metadata

Definition at line 103 of file larbatch/python/extractor_dict.py.

Constructor & Destructor Documentation

def python.extractor_dict.expMetaData.__init__ (   self,
  expname,
  inputfile 
)

Definition at line 105 of file larbatch/python/extractor_dict.py.

105 
106  def __init__(self, expname, inputfile):
107  MetaData.__init__(self, inputfile)
108  self.expname = expname
109  #self.exp_md_keyfile = expname + '_metadata_key'
110  try:
111  #translateMetaData = __import__("experiment_utilities", "MetaDataKey")
112  from experiment_utilities import MetaDataKey
113  except ImportError:
114  print("You have not defined an experiment-specific metadata and key-translating module in experiment_utilities. Exiting")
115  raise
116 
117  metaDataModule = MetaDataKey()
118  self.metadataList, self.translateKeyf = metaDataModule.metadataList(), metaDataModule.translateKey
print
do one_file $F done echo for F in find $TOP name CMakeLists txt print
Definition: update_sources.sh:94
python.extractor_dict.expMetaData.expname
expname
Definition: larbatch/python/extractor_dict.py:107
python.extractor_dict.expMetaData.__init__
def __init__
Definition: larbatch/python/extractor_dict.py:105
python.extractor_dict.expMetaData.translateKeyf
translateKeyf
Definition: larbatch/python/extractor_dict.py:117

Member Function Documentation

def python.extractor_dict.expMetaData.getmetadata (   self,
  md0 = {} 
) 
Get metadata from input file and return as python dictionary.
Calls other methods in class and returns metadata dictionary

Definition at line 220 of file larbatch/python/extractor_dict.py.

220 
221  def getmetadata(self, md0={}):
222  """ Get metadata from input file and return as python dictionary.
223  Calls other methods in class and returns metadata dictionary"""
224  proc = self.extract_metadata_to_pipe()
225  jobt = self.get_job(proc)
226  mdart = self.mdart_gen(jobt)
227  return self.md_gen(mdart, md0)
python.extractor_dict.expMetaData.getmetadata
def getmetadata
Definition: larbatch/python/extractor_dict.py:220
python.extractor_dict.expMetaData.md_gen
def md_gen
Definition: larbatch/python/extractor_dict.py:123
python.extractor_dict.MetaData.extract_metadata_to_pipe
def extract_metadata_to_pipe
Definition: larbatch/python/extractor_dict.py:41
python.extractor_dict.MetaData.mdart_gen
def mdart_gen
Definition: larbatch/python/extractor_dict.py:84
python.extractor_dict.MetaData.get_job
def get_job
Definition: larbatch/python/extractor_dict.py:55
def python.extractor_dict.expMetaData.md_gen (   self,
  mdart,
  md0 = {} 
) 
Loop through art metdata, generate metadata dictionary

Definition at line 123 of file larbatch/python/extractor_dict.py.

123 
124  def md_gen(self, mdart, md0={}):
125  """Loop through art metdata, generate metadata dictionary"""
126  # define an empty python dictionary which will hold sam metadata.
127  # Some fields can be copied directly from art metadata to sam metadata.
128  # Other fields require conversion.
129  md = {}
130 
131 
132 
133  # Loop over art metadata.
134  mixparents = []
135  for mdkey, mdval in list(mdart.items()):
136  # mdval = mdart[mdkey]
137 
138  # Skip some art-specific fields.
139  # Ignore primary run_type field (if any).
140  # Instead, get run_type from runs field.
141  if mdkey in ['file_format_version', 'file_format_era', 'run_type']:
142  pass
143  elif mdkey in ['art.file_format_version', 'art.file_format_era', 'art.run_type']:
144  pass
145 
146  # Ignore new-style first/last event (old-style first/last handled below).
147  elif mdkey in ['art.first_event', 'art.last_event']:
148  pass
149 
150  # Ignore data_stream if it begins with "out".
151  # These kinds of stream names are probably junk module labels.
152 
153  # First check if the data_stream is just "out" Catches an edge case
154  # where the stream does not have a number
155 
156  elif mdkey == 'data_stream' and mdval == 'out':
157  pass
158 
159  elif mdkey == 'data_stream' and mdval[:3] == 'out' and \
160  mdval[3] >= '0' and mdval[3] <= '9':
161  pass
162 
163  # Application family/name/version.
164  elif mdkey == 'applicationFamily' or mdkey == 'application.family':
165  md['application'], md['application']['family'] = self.md_handle_application(md), mdval
166  elif mdkey == 'process_name' or mdkey == 'art.process_name':
167  md['application'], md['application']['name'] = self.md_handle_application(md), mdval
168  elif mdkey == 'applicationVersion' or mdkey == 'application.version':
169  md['application'], md['application']['version'] = self.md_handle_application(md), mdval
170 
171  # Parents.
172  elif mdkey == 'parents':
173  md['parents'] = [{'file_name': parent} for parent in mdval]
174 
175  elif mdkey.startswith('mixparent'):
176  mixparents.append(mdval.strip(' ,"') )
177 
178  # Other fields where the key or value requires minor conversion.
179  elif mdkey in ['first_event', 'last_event']:
180  if (type(mdval) == type([]) or type(mdval) == type(())) and len(mdval) >= 3:
181  md[mdkey] = mdval[2]
182  else:
183  md[mdkey] = mdval
184  elif mdkey in self.metadataList:
185  #print mdkey
186  trkey = self.translateKey(mdkey)
187  #print trkey
188  md[trkey] = mdval
189  elif mdkey == 'fclName':
190  md['fcl.name'] = mdval
191  elif mdkey == 'fclVersion':
192  md['fcl.version'] = mdval
193 
194  #For all other keys, copy art metadata directly to sam metadata.
195  #This works for run-tuple (run, subrun, runtype) and time stamps.
196  else:
197  md[mdkey] = mdval
198 
199  # Merge mix parents into normal parents.
200 
201  for mixparent in mixparents:
202  mixparent_dict = {'file_name': mixparent}
203  md['parents'].append(mixparent_dict)
204 
205  # Get the other meta data field parameters.
206 
207  md['file_name'] = self.inputfile.split("/")[-1]
208  if 'file_size' in md0:
209  md['file_size'] = md0['file_size']
210  else:
211  md['file_size'] = os.path.getsize(self.inputfile)
212  if 'crc' in md0:
213  md['crc'] = md0['crc']
214  else:
215  md['crc'] = root_metadata.fileEnstoreChecksum(self.inputfile)
216 
217  # In case we ever want to check out what md is for any instance of MetaData by calling instance.md
218  self.md = md
219  return self.md
python.extractor_dict.expMetaData.md_gen
def md_gen
Definition: larbatch/python/extractor_dict.py:123
python.extractor_dict.MetaData.inputfile
inputfile
Definition: larbatch/python/extractor_dict.py:39
python.extractor_dict.expMetaData.md
md
Definition: larbatch/python/extractor_dict.py:217
python.extractor_dict.MetaData.md_handle_application
def md_handle_application
Definition: larbatch/python/extractor_dict.py:95
python.extractor_dict.expMetaData.translateKey
def translateKey
Definition: larbatch/python/extractor_dict.py:119
list
list
Definition: file_to_url.sh:28
def python.extractor_dict.expMetaData.translateKey (   self,
  key 
) 
Returns the output of the imported translateKey function (as translateKeyf) called on key

Definition at line 119 of file larbatch/python/extractor_dict.py.

119 
120  def translateKey(self, key):
121  """Returns the output of the imported translateKey function (as translateKeyf) called on key"""
122  return self.translateKeyf(key)
python.extractor_dict.expMetaData.translateKey
def translateKey
Definition: larbatch/python/extractor_dict.py:119
python.extractor_dict.expMetaData.translateKeyf
translateKeyf
Definition: larbatch/python/extractor_dict.py:117

Member Data Documentation

python.extractor_dict.expMetaData.expname

Definition at line 107 of file larbatch/python/extractor_dict.py.

python.extractor_dict.expMetaData.md

Definition at line 217 of file larbatch/python/extractor_dict.py.

python.extractor_dict.expMetaData.translateKeyf

Definition at line 117 of file larbatch/python/extractor_dict.py.

The documentation for this class was generated from the following file:
Generated by   doxygen 1.8.5