Package org.apache.sysds.runtime.instructions.spark.data

Class PartitionedBlock<T extends CacheBlock>

java.lang.Object

org.apache.sysds.runtime.instructions.spark.data.PartitionedBlock<T>

All Implemented Interfaces:

Externalizable, Serializable

public class PartitionedBlock<T extends CacheBlock>
extends Object
implements Externalizable

This class is for partitioned matrix/frame blocks, to be used as broadcasts. Distributed tasks require block-partitioned broadcasts but a lazy partitioning per task would create instance-local copies and hence replicate broadcast variables which are shared by all tasks within an executor.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
PartitionedBlock()
PartitionedBlock(int rlen, int clen, int blen)
PartitionedBlock(T block, int blen)
PartitionedBlock(T block, long[] dims, int blen)

Method Summary

Modifier and Type	Method	Description
void	clearBlocks()
PartitionedBlock<T>	createPartition(int offset, int numBlks)
T	getBlock(int[] ix)
T	getBlock(int rowIndex, int colIndex)
long	getBlocksize()
long	getDim(int i)
long	getExactSerializedSize()
long	getInMemorySize()
long	getNumCols()
int	getNumColumnBlocks()
int	getNumDimBlocks(int dim)
int	getNumRowBlocks()
long	getNumRows()
void	readExternal(ObjectInput is)	Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast deserialization.
void	setBlock(int rowIndex, int colIndex, T block)
void	writeExternal(ObjectOutput os)	Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast serialization.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

PartitionedBlock

public PartitionedBlock()

PartitionedBlock

public PartitionedBlock(T block,
 int blen)

PartitionedBlock

public PartitionedBlock(T block,
 long[] dims,
 int blen)

PartitionedBlock

public PartitionedBlock(int rlen,
 int clen,
 int blen)

Method Details

createPartition

public PartitionedBlock<T> createPartition(int offset,
 int numBlks)

getNumRows

public long getNumRows()

getNumCols

public long getNumCols()

getDim

public long getDim(int i)

getBlocksize

public long getBlocksize()

getNumRowBlocks

public int getNumRowBlocks()

getNumColumnBlocks

public int getNumColumnBlocks()

getNumDimBlocks

public int getNumDimBlocks(int dim)

getBlock

public T getBlock(int rowIndex,
 int colIndex)

getBlock

public T getBlock(int[] ix)

setBlock

public void setBlock(int rowIndex,
 int colIndex,
 T block)

getInMemorySize

public long getInMemorySize()

getExactSerializedSize

public long getExactSerializedSize()

clearBlocks

public void clearBlocks()

readExternal

public void readExternal(ObjectInput is)
                  throws IOException

Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast deserialization.

Specified by:

readExternal in interface Externalizable

Parameters:

is - object input

Throws:

IOException - if IOException occurs

writeExternal

public void writeExternal(ObjectOutput os)
                   throws IOException

Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast serialization.

Specified by:

writeExternal in interface Externalizable

Parameters:

os - object output

Throws:

IOException - if IOException occurs