FARGOS/VISTA Object Management Environment Core  ..
FARGOS/VISTA Object Management Environment Core Table of Contents
FARGOS I/O Processing Classes

The IO_Processor class is the driver of a suite of classes used to process streams of data. The OrderedInputSource class and its related classes add the ability to handle multiple streams of typically ordered-by-time streams data. The function processPacketFromSourceUsingClass() enables convenient processing of a stream of data by an object. More...

Modules

 Ordered Input/Output Streams
 The Ordered I/O facilities support simulation environments that consume time or sequence-ordered records from multiple sources.
 
 Memory-mapped Arrays of Variables
 Creates memory-mapped arrays of variables.
 
 Memory-Mapped File Functions
 Utility functions to create and open memory-mapped files.
 

Classes

struct  IO_metaBlock_header
 Structure to hold meta-data related to a block of received data. More...
 
class  IO_Processor_Statistics
 Holds collected statistics for an IO_Processor object. More...
 
class  IO_Processor
 Intermediary I/O processing object for performing multi-threaded receive-and-process operations on a flow of messages. More...
 
class  PollMonitor
 Monitor files and directories for changes. More...
 
class  ReadBinaryLog
 Reads a binary log, expands and formats as original plain text. If provided a text file, acts like 'cat'. More...
 
class  ReadMappedCircularBuffer
 Interface and generic implementation to traverse elements in a memory-mapped region under the control of a CircularBufferManager. More...
 
class  File_Buffer
 Implements an adjustable sliding buffer that minimizes data movement while enabling streams to process variable length content. More...
 
class  Read_And_Process_File
 Fundamental class that can read data from a memory-mapped region, file or socket and process data in lines, fixed-sized blocks or as data is available. More...
 
class  Extract_And_Process_Document_Stream
 Scan streams of data for start and end tags, invokes processDocument() on found content. More...
 
class  Parse_And_Process_HTTP_Stream
 Scan HTTP and parse stream for HTTP requests/responses. More...
 
class  CSV_File
 Read and process Comma-Separated-Value (or equivalent) files a record at a time. More...
 

Macros

#define OS_SOCKET_TYPE   int
 
#define OS_HANDLE_TYPE   int
 
#define INVALID_SOCKET   (-1)
 

Typedefs

typedef int(* IO_processBlockFP) (SharedBufferAllocRecord *rec, class IO_Processor *controller)
 Process a block of data received from an input source. More...
 
typedef ssize_t(* IO_receiveBlockFP) (SharedBufferAllocRecord *rec, class IO_Processor *controller)
 The receiveBlock routine should retrieve data and store it into the indicated block of memory, constrained by the blockLen. More...
 
typedef int(* IO_consumeFP) (class IO_Processor *controller)
 Long running routine which receives all input from input source. More...
 
typedef int(* IO_processFP) (class IO_Processor *controller)
 Long running routine which processes all input made available by the corresponding consume routine. More...
 
typedef int(* IO_blockAllocFP) (uint32_t blockLen, class IO_Processor *controller)
 
typedef int(* IO_blockFreeFP) (unsigned char *block, class IO_Processor *controller)
 

Functions

void setLabelForThread (pthread_t threadId, const char *label)
 
template<class PARSER_TYPE >
int processPacketUsingClass (SharedBufferAllocRecord *rec, class IO_Processor *controller)
 Implements a processing routine that invokes a processPacket() routine upon each incoming packet. More...
 
template<class PARSER_TYPE >
int processPacketFromSourceUsingClass (SharedBufferAllocRecord *rec, class IO_Processor *controller)
 Implements a processing routine that invokes a processPacketFromSource() routine upon each incoming packet, Differs from processPacketUsingClass() by exposing the meta data header. More...
 

Detailed Description

The IO_Processor class is the driver of a suite of classes used to process streams of data. The OrderedInputSource class and its related classes add the ability to handle multiple streams of typically ordered-by-time streams data. The function processPacketFromSourceUsingClass() enables convenient processing of a stream of data by an object.

Macro Definition Documentation

◆ INVALID_SOCKET

#define INVALID_SOCKET   (-1)

◆ OS_HANDLE_TYPE

#define OS_HANDLE_TYPE   int

◆ OS_SOCKET_TYPE

#define OS_SOCKET_TYPE   int

Typedef Documentation

◆ IO_blockAllocFP

typedef int(* IO_blockAllocFP) (uint32_t blockLen, class IO_Processor *controller)

◆ IO_blockFreeFP

typedef int(* IO_blockFreeFP) (unsigned char *block, class IO_Processor *controller)

◆ IO_consumeFP

typedef int(* IO_consumeFP) (class IO_Processor *controller)

Long running routine which receives all input from input source.

Return values
0should be returned as the value is currently ignored.

◆ IO_processBlockFP

typedef int(* IO_processBlockFP) (SharedBufferAllocRecord *rec, class IO_Processor *controller)

Process a block of data received from an input source.

Note
The processBlock routine should update the bytesProcessed counter of the statistics block.
Returns
an indicator if the block should be reclaimed or preserved.
Return values
0indicates that the block can be freed/reused. If 0 is not returned, then some provision must be made to reclaim the buffer block at a future point in time.

◆ IO_processFP

typedef int(* IO_processFP) (class IO_Processor *controller)

Long running routine which processes all input made available by the corresponding consume routine.

Return values
0should be returned as the value is currently ignored.

◆ IO_receiveBlockFP

typedef ssize_t(* IO_receiveBlockFP) (SharedBufferAllocRecord *rec, class IO_Processor *controller)

The receiveBlock routine should retrieve data and store it into the indicated block of memory, constrained by the blockLen.

Returns
The number of bytes read is returned.
Return values
0indicates end-of-file.
-1indicates an error.

Function Documentation

◆ processPacketFromSourceUsingClass()

template<class PARSER_TYPE >
int processPacketFromSourceUsingClass ( SharedBufferAllocRecord rec,
class IO_Processor controller 
)
inline

Implements a processing routine that invokes a processPacketFromSource() routine upon each incoming packet, Differs from processPacketUsingClass() by exposing the meta data header.

References IO_Processor::_PACKET_HAS_META_DATA, IO_Processor::blockingMode, IO_Processor::bufferAddress(), IO_Processor::bufferHeaderAddress(), IO_Processor::getExtraData(), OME_EXPECT_FALSE, and SharedBufferAllocRecord_32::usedLen.

◆ processPacketUsingClass()

template<class PARSER_TYPE >
int processPacketUsingClass ( SharedBufferAllocRecord rec,
class IO_Processor controller 
)
inline

Implements a processing routine that invokes a processPacket() routine upon each incoming packet.

References IO_Processor::bufferAddress(), IO_Processor::getExtraData(), OME_EXPECT_FALSE, and SharedBufferAllocRecord_32::usedLen.

◆ setLabelForThread()

void setLabelForThread ( pthread_t  threadId,
const char *  label 
)
Generated: Tue Jul 28 2020 16:03:26
Support Information