Struct sparrow::scanner::buffer_loader::BufferLoader

pub struct BufferLoader {
    pub size: usize,
    pub base_model: Tree,
    pub base_model_sig: String,
    pub current_version: usize,
    pub new_buffer: Arc<RwLock<Option<(usize, Vec<(LabeledData<u8, i8>, (f32, usize))>, Tree, String)>>>,
    pub sample_mode: SampleMode,
    pub ess: f32,
    // some fields omitted
}

Double-buffered sample set. It consists of two buffers stores in memory. One of the buffer is used for providing examples to the boosting algorithm, meanwhile the other would receive new examples from the sampler module. Once the other buffer is filled, the functions of the two buffers switch so that the sample set used for training would be updated periodically.

It might take several scans over the sample set before the new sample set is ready. Thus it is very likely that the examples in the sample set would be accessed multiple times.

Fields

size: usize

base_model: Tree

base_model_sig: String

current_version: usize

new_buffer: Arc<RwLock<Option<(usize, Vec<(LabeledData<u8, i8>, (f32, usize))>, Tree, String)>>>

sample_mode: SampleMode

ess: f32

Methods

impl BufferLoader

pub fn new(
    size: usize,
    batch_size: usize,
    sample_mode: SampleMode,
    sleep_duration: usize,
    min_ess: f32,
    exp_name: String
) -> BufferLoader

Create a new BufferLoader.

size: the size of sample set. The total size of the two buffers would be 2x of size. batch_size: the number of examples that feeds to the boosting algorithm at a time.

If init_block is set to true, this function would block until the first sample set is created (i.e. enough examples is received from the sampler). Alternatively, one can explicitly call the load function to create the first sample set before start training the boosting algorithm.

pub fn get_num_batches(&self) -> usize

Return the number of batches (i.e. the number of function calls to get_next_batch) before exhausting the current sample set.

pub fn get_next_batch(
    &mut self,
    allow_switch: bool
) -> (&[(LabeledData<u8, i8>, (f32, f32, usize, usize))], bool)

Read next batch of examples.

allow_switch: If set to true, the alternate buffer would be checked if it was ready. If it was, the loader will switched to the alternate buffer for reading the next batch of examples.

pub fn get_next_batch_and_update(
    &mut self,
    allow_switch: bool,
    model: &Tree
) -> (&[(LabeledData<u8, i8>, (f32, f32, usize, usize))], bool)

pub fn try_switch(&mut self) -> bool

pub fn check_ess_blocking(&mut self)

pub fn reset_scores(&mut self)

pub fn is_empty(&self) -> bool