FARGOS/VISTA Object Management Environment Core
..
|
Implements an adjustable sliding buffer that minimizes data movement while enabling streams to process variable length content. More...
#include <read_file.hpp>
Public Types | |
enum | { EXTERNAL_SEGMENT = (1 << 0), IS_READ_ONLY = (1 << 1) } |
Public Member Functions | |
File_Buffer (size_t bfrSize=65536) | |
Creates an empty buffer of the indicated size in the heap. More... | |
File_Buffer (unsigned char *existingBuffer, size_t bfrSize, size_t currentLen=0, uint8_t ownershipFlags=EXTERNAL_SEGMENT) | |
Accepts the use (and potentially transfers ownership) of an existing buffer that may already have some content. More... | |
~File_Buffer () | |
void | useRegionAsBuffer (unsigned char *existingBuffer, size_t bfrSize, size_t currentLen, uint8_t ownershipFlags) OME_ALWAYS_INLINE OME_ALWAYS_OPTIMIZE("-O3") |
Replace the existing buffer with a new region. More... | |
size_t | spaceLeft () const OME_ALWAYS_INLINE |
Returns the amount of unused space in the buffer. More... | |
size_t | spaceUsed () const OME_ALWAYS_INLINE |
Returns the amount of space current used in the buffer. More... | |
unsigned char * | startBufferLocation () const OME_ALWAYS_INLINE |
Returns the start of the active content in the buffer. More... | |
unsigned char * | endBufferLocation () const OME_ALWAYS_INLINE |
Returns just past the end of the active content in the buffer. Normally, this would be where new content would be appended. More... | |
void | adjustBufferEnd (size_t amount) OME_ALWAYS_INLINE |
Adjusts the end of the buffer. This is typically called after new content has been added. More... | |
void | adjustBufferStart (size_t amount) OME_ALWAYS_INLINE |
Adjusts the start of the active content in the buffer. More... | |
void | discardInitialBytes (size_t bytes=0) OME_ALWAYS_INLINE OME_ALWAYS_OPTIMIZE("-O3") |
Discards bytes from the beginning of the active portion of the buffer; typically as a result of having processed them. More... | |
size_t | appendDataToBuffer (const unsigned char *newData, size_t bytesToAdd, size_t startFromOffset=0) OME_ALWAYS_INLINE OME_ALWAYS_OPTIMIZE("-O3") |
Convenience routine to append data to the buffer. More... | |
Public Attributes | |
unsigned char * | buffer |
base of the buffer segment More... | |
size_t | bufferLen |
total length of buffer segment More... | |
size_t | endBufferOffset |
offset of last byte used in buffer More... | |
size_t | startBufferOffset |
offset of first byte used in buffer More... | |
uint8_t | externalBufferFlags |
flags indicating read-only, external More... | |
Implements an adjustable sliding buffer that minimizes data movement while enabling streams to process variable length content.
Typical usage patterns have an outer loop whose goal is to make calls to appendDataToBuffer() until all of the data has been appended. Within that loop, it is typical to first test if spaceLeft() is less than the amount of new data to be added. If so, a call to discardInitialBytes(0) is made in an attempt to free up all previously handled content in the front of the buffer. Next, a call to appendDataToBuffer() is made, which returns the number of bytes that were successfully appended. That result is tested to see if the number of bytes actually appended was the same as requested quantity. If not, depending on the application, this could be an immediately fatal error. For other scenarios, a sufficient quantity might now be in the buffer to allow intermediate records to be successfully processed. In those applications, there is an inner loop that processes all complete records that are in the buffer. Once those records have been processed, their corresponding data can be discarded from the buffer, which in turn would free up space to allow more of the remaining content to be appended on another pass of the outer loop.
anonymous enum |
|
inlineexplicit |
Creates an empty buffer of the indicated size in the heap.
References buffer, bufferLen, endBufferOffset, externalBufferFlags, and startBufferOffset.
|
inline |
Accepts the use (and potentially transfers ownership) of an existing buffer that may already have some content.
existingBuffer | points to the existing buffer. |
bfrSize | indicates the size of the existing buffer. |
currentLen | specifies the length of the current content in the buffer. |
ownershipFlags | specifies a mask of ownership attributes. If the EXTERNAL_SEGMENT bit is set, the buffer will not be deleted. This arises when the region is owned by another object or is on the stack. If the IS_READ_ONLY bit is set, no attempts to modify the buffer will be honored. This is useful when the region is a read-only memory-mapped file that should not be modified. |
References buffer, bufferLen, endBufferOffset, externalBufferFlags, and startBufferOffset.
|
inline |
References buffer, EXTERNAL_SEGMENT, externalBufferFlags, and IS_READ_ONLY.
|
inline |
Adjusts the end of the buffer. This is typically called after new content has been added.
amount | specifies the number of bytes to be added. |
References endBufferOffset.
Referenced by Read_And_Process_File::readIntoFileBuffer().
|
inline |
Adjusts the start of the active content in the buffer.
amount | specifies the number of bytes to be dropped from the beginning of the active portion of the buffer. |
For efficiency, normally only the start of the buffer is adjusted rather than performing a block move to reorient the data in the buffer.
See discardInitialBytes() for the routine that is used to free up needed space.
References startBufferOffset.
Referenced by Read_And_Process_File::findAndProcessNextLine(), ReadBinaryLog::processBuffer(), and Read_And_Process_File::readAndProcessBlocksFromFile().
|
inline |
Convenience routine to append data to the buffer.
newData | points to the content to be added. |
bytesToAdd | specifies the length of the content to be added. |
startFromOffset | specifies an optional offset relative to newData. |
The effective source address of the new content is newData + offset.
NOTE: this routine does not call discardInitialBytes() to free up space on demand; this is the responsibility of the caller.
References buffer, endBufferOffset, externalBufferFlags, IS_READ_ONLY, OME_EXPECT_FALSE, spaceLeft(), spaceUsed(), and useRegionAsBuffer().
Referenced by Extract_And_Process_Document_Stream::addIOblockThenProcess(), CSV_File::processPacket(), and Extract_And_Process_Document_Stream::scanAndProcessBuffer().
|
inline |
Discards bytes from the beginning of the active portion of the buffer; typically as a result of having processed them.
References buffer, endBufferOffset, externalBufferFlags, IS_READ_ONLY, OME_EXPECT_FALSE, spaceUsed(), and startBufferOffset.
Referenced by Extract_And_Process_Document_Stream::addIOblockThenProcess(), ReadBinaryLog::processBuffer(), CSV_File::processPacket(), Read_And_Process_File::readAndProcessBlocksFromFile(), Read_And_Process_File::readAndProcessTextLines(), Extract_And_Process_Document_Stream::scanAndProcessBuffer(), and Extract_And_Process_Document_Stream::scanForDocument().
|
inline |
Returns just past the end of the active content in the buffer. Normally, this would be where new content would be appended.
References buffer, and endBufferOffset.
Referenced by Read_And_Process_File::readIntoFileBuffer().
|
inline |
Returns the amount of unused space in the buffer.
References bufferLen, and endBufferOffset.
Referenced by Extract_And_Process_Document_Stream::addIOblockThenProcess(), appendDataToBuffer(), Read_And_Process_File::readIntoFileBuffer(), and Extract_And_Process_Document_Stream::scanAndProcessBuffer().
|
inline |
Returns the amount of space current used in the buffer.
References endBufferOffset, and startBufferOffset.
Referenced by appendDataToBuffer(), discardInitialBytes(), Read_And_Process_File::findAndProcessNextLine(), ReadBinaryLog::processBuffer(), Read_And_Process_File::readAndProcessBlocksFromFile(), Read_And_Process_File::readAndProcessTextLines(), Extract_And_Process_Document_Stream::scanAndProcessBuffer(), and Extract_And_Process_Document_Stream::scanForDocument().
|
inline |
Returns the start of the active content in the buffer.
References buffer, and startBufferOffset.
Referenced by Read_And_Process_File::findAndProcessNextLine(), ReadBinaryLog::processBuffer(), Read_And_Process_File::readAndProcessBlocksFromFile(), Extract_And_Process_Document_Stream::scanAndProcessBuffer(), and Extract_And_Process_Document_Stream::scanForDocument().
|
inline |
Replace the existing buffer with a new region.
existingBuffer | points to the existing buffer. |
bfrSize | indicates the size of the existing buffer. |
currentLen | specifies the length of the current content in the buffer. |
ownershipFlags | specifies a mask of ownership attributes. If the EXTERNAL_SEGMENT bit is set, the buffer will not be deleted. If the IS_READ_ONLY bit is set, no attempts to modify the buffer will be honored. |
NOTE: should only be used if spaceUsed() is 0. Obviously, caution should be used to ensure no entity expects the buffer region to remain unchanged.
References buffer, bufferLen, endBufferOffset, EXTERNAL_SEGMENT, externalBufferFlags, IS_READ_ONLY, and startBufferOffset.
Referenced by appendDataToBuffer().
unsigned char* File_Buffer::buffer |
base of the buffer segment
Referenced by appendDataToBuffer(), discardInitialBytes(), endBufferLocation(), File_Buffer(), startBufferLocation(), useRegionAsBuffer(), and ~File_Buffer().
size_t File_Buffer::bufferLen |
total length of buffer segment
Referenced by File_Buffer(), spaceLeft(), and useRegionAsBuffer().
size_t File_Buffer::endBufferOffset |
offset of last byte used in buffer
Referenced by adjustBufferEnd(), appendDataToBuffer(), discardInitialBytes(), endBufferLocation(), File_Buffer(), spaceLeft(), spaceUsed(), and useRegionAsBuffer().
uint8_t File_Buffer::externalBufferFlags |
flags indicating read-only, external
Referenced by appendDataToBuffer(), discardInitialBytes(), File_Buffer(), useRegionAsBuffer(), and ~File_Buffer().
size_t File_Buffer::startBufferOffset |
offset of first byte used in buffer
Referenced by adjustBufferStart(), discardInitialBytes(), File_Buffer(), spaceUsed(), startBufferLocation(), and useRegionAsBuffer().
![]() | Generated: Tue Jul 28 2020 16:03:27
Support Information |