icarus::details::KeyedCSVparser Class Reference

Parser to fill a KeyValuesData structure out of a character buffer. More...

#include <KeyedCSVparser.h>

Classes
struct	InvalidFormat
struct	MissingValues
struct	ParserError

Public Types
using	ParsedData_t = icarus::KeyValuesData
using	Error = icarus::KeyValuesData::Error
	Base of all errors by KeyedCSVparser. More...
using	ErrorOnKey = icarus::KeyValuesData::ErrorOnKey
using	MissingSize = KeyValuesData::MissingSize
	Expected number of values is missing. More...

Public Member Functions
	KeyedCSVparser (char sep= ',')
	Constructor: specifies the separator character. More...
template<typename String >
auto	makeBuffer (String const &s) noexcept-> Buffer_t
template<char... Chars>
auto	stripRightChars (SubBuffer_t s) noexcept-> SubBuffer_t
template<typename BIter , typename EIter >
auto	parse (BIter b, EIter e) const -> ParsedData_t
ParsedData_t	parse (std::string_view const &s) const
	Parses the buffer s and returns a data structure with the content. More...
ParsedData_t	parse (std::string const &s) const
template<typename BIter , typename EIter >
ParsedData_t	parse (BIter b, EIter e) const
ParsedData_t	operator() (std::string_view const &s) const
ParsedData_t	operator() (std::string const &s) const
template<typename BIter , typename EIter >
ParsedData_t	operator() (BIter b, EIter e) const
void	parse (std::string_view const &s, ParsedData_t &data) const
	Parses the buffer s and fills data with it. More...
Know patterns
The parser normally treats as a value everything that does not start with a letter. Known patterns may override this behaviour: if a token matches a known pattern, it is considered a key and it is possible to specify the expected number of values. The number of values can be: a number: exactly that number of values are required; an exception will be thrown if not enough tokens are available; FixedSize: the next token must be a non-negative integer specifying how many other values to add (read this with Item::getSizedVector()); an exception will be thrown if not enough tokens are available; DynamicSize: the standard algorithm is used and values are added as long as they don't look like keys; the token matching the pattern is interpreted as a key though. Patterns are considered in the order they were added.
KeyedCSVparser &	addPattern (std::regex pattern, unsigned int values)
	Adds a single known pattern. More...
KeyedCSVparser &	addPattern (std::string const &pattern, unsigned int values)
KeyedCSVparser &	addPatterns (std::initializer_list< std::pair< std::regex, unsigned int >> patterns)
	Adds known patterns. More...
KeyedCSVparser &	addPatterns (std::initializer_list< std::pair< std::string, unsigned int >> patterns)

Static Public Attributes
static constexpr unsigned int	FixedSize = std::numeric_limits<unsigned int>::max()
	Expected values are missing. More...
static constexpr unsigned int	DynamicSize = FixedSize - 1U
	Mnemonic size value used in addPattern() calls. More...

Private Types
using	Buffer_t = std::string_view
using	SubBuffer_t = std::string_view

Private Member Functions
std::size_t	findTokenLength (Buffer_t const &buffer) const noexcept
	Returns the length of the next toke, up to the next separator (excluded). More...
SubBuffer_t	peekToken (Buffer_t const &buffer) const noexcept
	Returns the value of the next token, stripped. More...
SubBuffer_t	extractToken (Buffer_t &buffer) const noexcept
	Extracts the next token from the buffer and returns its value, stripped. More...
bool	isKey (SubBuffer_t const &buffer) const noexcept
	Is content of buffer a key (as opposed to a value)? More...

Static Private Member Functions
template<typename String >
static Buffer_t	makeBuffer (String const &s) noexcept
static Buffer_t &	moveBufferHead (Buffer_t &buffer, std::size_t size) noexcept
static SubBuffer_t	strip (SubBuffer_t s) noexcept
static SubBuffer_t	stripLeft (SubBuffer_t s) noexcept
static SubBuffer_t	stripRight (SubBuffer_t s) noexcept
static SubBuffer_t	stripRightChar (SubBuffer_t s, char c) noexcept
template<char... Chars>
static SubBuffer_t	stripRightChars (SubBuffer_t s) noexcept

Private Attributes
char const	fSep = ','
	Character used as token separator. More...
std::vector< std::pair < std::regex, unsigned int > >	fPatterns
	List of known patterns for matching keys, and how many values they hold. More...

Detailed Description

Parser to fill a KeyValuesData structure out of a character buffer.

It currently supports only single-line buffer.

The parser operates one "line" at a time, returning a KeyValuesData with the values assigned to each detected key. No data type is implied: all elements are treated as strings, either a key or a value. The parser separates the elements according to a separator, strips them of trailing and heading spaces, then it decides whether each element is a value to be assigned to the last key found, or a new key. Keys are elements that have letters in them, values are anything else. This simple (and arguable) criterion can be broken with specific parser configuration: a pattern can be specified that when matched to an element will make it a key; the pattern can also set the number of values that key will require.

For example:

icarus::details::KeyedCSVparser parser;
parser.addPatterns({
      { "TriggerType", 1U } // expect one value (even if contains letters)
    , { "TriggerWindows", 1U } // expect one value (even if contains letters)
    , { "TPChitTimes", icarus::details::KeyedCSVparser::FixedSize }
         // the first value is an integer, count of how many other values
  });

icarus::KeyValuesData data = parser(
  "TriggerType, S5, Triggers, TriggerWindows, 0C0B,"
  " TPChits, 12, 130, 0, 0, TPChitTimes, 3, -1.1, -0.3, 0.1, PMThits, 8"
  );

will return data with 6 items.

Definition at line 68 of file KeyedCSVparser.h.

Member Typedef Documentation

using icarus::details::KeyedCSVparser::Buffer_t = std::string_view

private

Definition at line 164 of file KeyedCSVparser.h.

using icarus::details::KeyedCSVparser::Error = icarus::KeyValuesData::Error

Base of all errors by KeyedCSVparser.

Definition at line 75 of file KeyedCSVparser.h.

using icarus::details::KeyedCSVparser::ErrorOnKey = icarus::KeyValuesData::ErrorOnKey

Definition at line 76 of file KeyedCSVparser.h.

using icarus::details::KeyedCSVparser::MissingSize = KeyValuesData::MissingSize

Expected number of values is missing.

Parsing format is not understood.

Definition at line 80 of file KeyedCSVparser.h.

using icarus::details::KeyedCSVparser::ParsedData_t = icarus::KeyValuesData

Definition at line 72 of file KeyedCSVparser.h.

using icarus::details::KeyedCSVparser::SubBuffer_t = std::string_view

private

Definition at line 165 of file KeyedCSVparser.h.

Constructor & Destructor Documentation

icarus::details::KeyedCSVparser::KeyedCSVparser ( char  sep = ',' )

inline

Constructor: specifies the separator character.

Definition at line 91 of file KeyedCSVparser.h.

: fSep(sep) {}

Member Function Documentation

KeyedCSVparser& icarus::details::KeyedCSVparser::addPattern ( std::regex  pattern, unsigned int  values  )

inline

Adds a single known pattern.

Parameters

pattern	the regular expression matching the key for this pattern
values	the number of values for this pattern

Returns

this parser (addPattern() calls may be chained)

Definition at line 141 of file KeyedCSVparser.h.

    { fPatterns.emplace_back(std::move(pattern), values); return *this; }

KeyedCSVparser& icarus::details::KeyedCSVparser::addPattern ( std::string const &  pattern, unsigned int  values  )

inline

Definition at line 143 of file KeyedCSVparser.h.

    { return addPattern(std::regex{ pattern }, values); }

auto icarus::details::KeyedCSVparser::addPatterns

(

std::initializer_list< std::pair< std::regex, unsigned int >>

patterns

)

Adds known patterns.

Parameters

patterns

sequence of patterns to be added

Returns

this parser (addPatterns() calls may be chained)

Each pattern is a pair key regex/number of values, like in addPattern().

Definition at line 114 of file KeyedCSVparser.cxx.

{
  for (auto& pattern: patterns) fPatterns.emplace_back(std::move(pattern));
  return *this;
} // icarus::details::KeyedCSVparser::addPatterns()

auto icarus::details::KeyedCSVparser::addPatterns

(

std::initializer_list< std::pair< std::string, unsigned int >>

patterns

)

Definition at line 124 of file KeyedCSVparser.cxx.

{
  for (auto& pattern: patterns)
    fPatterns.emplace_back(std::regex{ pattern.first }, pattern.second);
  return *this;
} // icarus::details::KeyedCSVparser::addPatterns()

auto icarus::details::KeyedCSVparser::extractToken ( Buffer_t &  buffer ) const

privatenoexcept

Extracts the next token from the buffer and returns its value, stripped.

Definition at line 165 of file KeyedCSVparser.cxx.

{
#if 1
  auto const start = cbegin(buffer), bend = cend(buffer);
  std::size_t const length = findTokenLength(buffer);
  moveBufferHead(buffer, length + ((start + length == bend)? 0: 1));
  return strip({ start, length });
#else
  
  auto const start = cbegin(buffer), bend = cend(buffer);
  auto finish = start;
  while (finish != bend) {
    if (*finish == fSep) break;
    ++finish;
  } // for
  
  // update the start of the buffer
  std::size_t const tokenLength = std::distance(start, finish);
  moveBufferHead(buffer, tokenLength + ((finish == bend)? 0: 1));
  
  return strip({ start, tokenLength });
#endif
} // icarus::details::KeyedCSVparser::extractToken()

std::size_t icarus::details::KeyedCSVparser::findTokenLength ( Buffer_t const &  buffer ) const

privatenoexcept

Returns the length of the next toke, up to the next separator (excluded).

Definition at line 141 of file KeyedCSVparser.cxx.

{
  
  auto const start = cbegin(buffer), bend = cend(buffer);
  auto finish = start;
  while (finish != bend) {
    if (*finish == fSep) break;
    ++finish;
  } // for
  
  return std::distance(start, finish);
} // icarus::details::KeyedCSVparser::findTokenLength()

bool icarus::details::KeyedCSVparser::isKey ( SubBuffer_t const &  buffer ) const

privatenoexcept

Is content of buffer a key (as opposed to a value)?

Definition at line 192 of file KeyedCSVparser.cxx.

{
  
  return !buffer.empty() && std::isalpha(buffer.front());
  
} // icarus::details::KeyedCSVparser::isKey()

template<typename String >

static Buffer_t icarus::details::KeyedCSVparser::makeBuffer ( String const &  s )

staticprivatenoexcept

template<typename String >

auto icarus::details::KeyedCSVparser::makeBuffer ( String const &  s ) -> Buffer_t

noexcept

Definition at line 202 of file KeyedCSVparser.cxx.

  { return { data(s), size(s) }; } // C++20: use begin/end constructor

auto icarus::details::KeyedCSVparser::moveBufferHead ( Buffer_t &  buffer, std::size_t  size  )

staticprivatenoexcept

Definition at line 210 of file KeyedCSVparser.cxx.

{
  
  size = std::min(size, buffer.size());
  return buffer = { buffer.data() + size, buffer.size() - size };
  
} // details::KeyedCSVparser::eatBufferHead()

ParsedData_t icarus::details::KeyedCSVparser::operator() ( std::string_view const &  s ) const

inline

Definition at line 100 of file KeyedCSVparser.h.

{ return parse(s); }

ParsedData_t icarus::details::KeyedCSVparser::operator() ( std::string const &  s ) const

inline

Definition at line 101 of file KeyedCSVparser.h.

{ return parse(s); }

template<typename BIter , typename EIter >

ParsedData_t icarus::details::KeyedCSVparser::operator() ( BIter  b, EIter  e  ) const

inline

Definition at line 103 of file KeyedCSVparser.h.

{ return parse(b, e); }

auto icarus::details::KeyedCSVparser::parse ( std::string_view const &  s ) const

inline

Parses the buffer s and returns a data structure with the content.

Definition at line 236 of file KeyedCSVparser.h.

  { ParsedData_t data; parse(s, data); return data; }

auto icarus::details::KeyedCSVparser::parse

(

std::string const &

s

)

const

Definition at line 135 of file KeyedCSVparser.cxx.

  { return parse(std::string_view{ s.data(), s.size() }); }

template<typename BIter , typename EIter >

ParsedData_t icarus::details::KeyedCSVparser::parse	(	BIter	b,
		EIter	e
	)		const

void icarus::details::KeyedCSVparser::parse	(	std::string_view const &	s,
		ParsedData_t &	data
	)		const

Parses the buffer s and fills data with it.

Definition at line 22 of file KeyedCSVparser.cxx.

{
  
  auto stream = s;
  
  ParsedData_t::Item* currentItem = nullptr;
  
  // this many tokens will be assigned to the current key:
  int forcedValues = -1; // 0 would force the first entry to be a key
  
  while (!stream.empty()) {
    
    auto const token = extractToken(stream);
    
    std::string tokenStr { cbegin(token), cend(token) };
    
    bool bKey = false;
    do {
      
      // if there are values pending, this is not a key, period;
      // if all required values have been assigned, the next token is a key.
      if (forcedValues >= 0) {
        bKey = (forcedValues == 0); // if no more forced values, next is key
        --forcedValues;
        // if we know the token is a key, we still need to check for matching
        // patterns to assign required values
        if (!bKey) break;
      }
      
      // the token may still be a key (if `bKey` is true, it is for sure: we can
      // decide that a non-key (!bKey) is actually a key, but not the opposite)
      for (auto const& [ pattern, values ]: fPatterns) {
        if (!std::regex_match(begin(token), end(token), pattern)) continue;
        bKey = true; // matching a pattern implies this is a key
        std::string const& key = tokenStr;
        // how many values to expect:
        switch (values) {
          case FixedSize: // read the next token immediately as fixed size
            {
              if (stream.empty()) throw MissingSize(key);
              
              auto const sizeToken = peekToken(stream);
              if (empty(sizeToken)) throw MissingSize(key);
              
              // the value is loaded in `forcedValues` and already excludes
              // the size token just read
              char const *b = begin(sizeToken), *e = end(sizeToken);
              if (std::from_chars(b, e, forcedValues).ptr != e)
                throw MissingSize(key, std::string{ sizeToken });
              
              ++forcedValues; // the size will be forced in the values anyway
              
            } // FixedSize
            break;
          case DynamicSize:
            // nothing to do, the normal algorithm rules will follow
            break;
          default:
            forcedValues = values;
            break;
        } // switch
        break;
      } // for pattern
      if (bKey) break;
      
      // let the "standard" pattern decide
      bKey = isKey(token);
      
    } while (false);
    
    if (bKey) currentItem = &(data.makeItem(std::move(tokenStr)));
    else {
      if (!currentItem) {
        throw InvalidFormat(
         "values started without a key ('" + tokenStr + "' is not a valid key)."
         );
      }
      currentItem->addValue(std::move(tokenStr));
    }
    
  } // while
  
  if (forcedValues > 0) {
    assert(currentItem);
    throw MissingValues(currentItem->key(), forcedValues);
  }
  
} // icarus::KeyedCSVparser::parse()

template<typename BIter , typename EIter >

auto icarus::details::KeyedCSVparser::parse	(	BIter	b,
		EIter	e
	)		const -> ParsedData_t

Definition at line 244 of file KeyedCSVparser.h.

  { return parse(std::string_view{ &*b, std::distance(b, e) }); }

auto icarus::details::KeyedCSVparser::peekToken ( Buffer_t const &  buffer ) const

privatenoexcept

Returns the value of the next token, stripped.

Definition at line 157 of file KeyedCSVparser.cxx.

{
  return strip({ cbegin(buffer), findTokenLength(buffer) });
} // icarus::details::KeyedCSVparser::peekToken()

auto icarus::details::KeyedCSVparser::strip ( SubBuffer_t  s )

staticprivatenoexcept

Definition at line 220 of file KeyedCSVparser.cxx.

  { return stripRight(stripLeft(stripRightChars<'\n', '\r', '\0'>(s))); }

auto icarus::details::KeyedCSVparser::stripLeft ( SubBuffer_t  s )

staticprivatenoexcept

Definition at line 226 of file KeyedCSVparser.cxx.

{
  
  while (!s.empty()) {
    if (!std::isspace(s.front())) break;
    s.remove_prefix(1);
  }
  return s;
  
} // icarus::details::KeyedCSVparser::stripLeft()

auto icarus::details::KeyedCSVparser::stripRight ( SubBuffer_t  s )

staticprivatenoexcept

Definition at line 240 of file KeyedCSVparser.cxx.

{
  
  while (!s.empty()) {
    if (!std::isspace(s.back())) break;
    s.remove_suffix(1);
  }
  return s;
  
} // icarus::details::KeyedCSVparser::stripRight()

auto icarus::details::KeyedCSVparser::stripRightChar ( SubBuffer_t  s, char  c  )

staticprivatenoexcept

Definition at line 255 of file KeyedCSVparser.cxx.

{
  
  while (!s.empty()) {
    if (s.back() != c) break;
    s.remove_suffix(1);
  }
  return s;
  
} // icarus::details::KeyedCSVparser::stripRightChar()

template<char... Chars>

static SubBuffer_t icarus::details::KeyedCSVparser::stripRightChars ( SubBuffer_t  s )

staticprivatenoexcept

template<char... Chars>

auto icarus::details::KeyedCSVparser::stripRightChars ( SubBuffer_t  s ) -> SubBuffer_t

noexcept

Definition at line 270 of file KeyedCSVparser.cxx.

{
  while (true) {
    auto ns = s;
    for (char c: { Chars... }) ns = stripRightChar(ns, c);
    if (ns == s) return ns;
    s = ns;
  } // while(true)
  
} // icarus::details::KeyedCSVparser::stripRightChars()

Member Data Documentation

constexpr unsigned int icarus::details::KeyedCSVparser::DynamicSize = FixedSize - 1U

static

Mnemonic size value used in addPattern() calls.

Definition at line 87 of file KeyedCSVparser.h.

constexpr unsigned int icarus::details::KeyedCSVparser::FixedSize = std::numeric_limits<unsigned int>::max()

static

Expected values are missing.

Mnemonic size value used in addPattern() calls.

Definition at line 85 of file KeyedCSVparser.h.

std::vector<std::pair<std::regex, unsigned int> > icarus::details::KeyedCSVparser::fPatterns

private

List of known patterns for matching keys, and how many values they hold.

Definition at line 170 of file KeyedCSVparser.h.

char const icarus::details::KeyedCSVparser::fSep = ','

private

Character used as token separator.

Definition at line 167 of file KeyedCSVparser.h.

---

The documentation for this class was generated from the following files:

• KeyedCSVparser.h
• KeyedCSVparser.cxx