Package org.apache.sysds.runtime.transform.encode

Class ColumnEncoderComposite

java.lang.Object
org.apache.sysds.runtime.transform.encode.ColumnEncoder
org.apache.sysds.runtime.transform.encode.ColumnEncoderComposite
All Implemented Interfaces:
Externalizable, Serializable, Comparable<ColumnEncoder>, Encoder
public class ColumnEncoderComposite extends ColumnEncoder
Simple composite encoder that applies a list of encoders in specified order. By implementing the default encoder API it can be used as a drop-in replacement for any other encoder.
See Also:

  • Constructor Details

    • ColumnEncoderComposite

      public ColumnEncoderComposite()

    • ColumnEncoderComposite

      public ColumnEncoderComposite(List<ColumnEncoder> columnEncoders, FrameBlock meta)

    • ColumnEncoderComposite

      public ColumnEncoderComposite(List<ColumnEncoder> columnEncoders)

    • ColumnEncoderComposite

      public ColumnEncoderComposite(List<ColumnEncoder> columnEncoders, int colID)

    • ColumnEncoderComposite

      public ColumnEncoderComposite(ColumnEncoder columnEncoder)

  • Method Details

    • getEncoders

      public List<ColumnEncoder> getEncoders()

    • getEncoder

      public <T extends ColumnEncoder> T getEncoder(Class<T> type)

    • isEncoder

      public boolean isEncoder(int colID, Class<?> type)

    • build

      public void build(CacheBlock<?> in)
      Description copied from interface: Encoder
      Build the transform meta data for the given block input. This call modifies and keeps meta data as encoder state.
      Parameters:
      in - input frame block

    • build

      public void build(CacheBlock<?> in, Map<Integer,double[]> equiHeightMaxs)
      Overrides:
      build in class ColumnEncoder

    • getApplyTasks

      public List<DependencyTask<?>> getApplyTasks(CacheBlock<?> in, MatrixBlock out, int outputCol, int[] sparseRowPointerOffsets)
      Overrides:
      getApplyTasks in class ColumnEncoder

    • getBuildTasks

      public List<DependencyTask<?>> getBuildTasks(CacheBlock<?> in)
      Overrides:
      getBuildTasks in class ColumnEncoder

    • prepareBuildPartial

      public void prepareBuildPartial()
      Description copied from class: ColumnEncoder
      Allocates internal data structures for partial build.
      Specified by:
      prepareBuildPartial in interface Encoder
      Overrides:
      prepareBuildPartial in class ColumnEncoder

    • buildPartial

      public void buildPartial(FrameBlock in)
      Description copied from class: ColumnEncoder
      Partial build of internal data structures (e.g., in distributed spark operations).
      Specified by:
      buildPartial in interface Encoder
      Overrides:
      buildPartial in class ColumnEncoder
      Parameters:
      in - input frame block

    • apply

      public MatrixBlock apply(CacheBlock<?> in, MatrixBlock out, int outputCol, int rowStart, int blk)
      Overrides:
      apply in class ColumnEncoder

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • mergeAt

      public void mergeAt(ColumnEncoder other)
      Description copied from class: ColumnEncoder
      Merges another encoder, of a compatible type, in after a certain position. Resizes as necessary. ColumnEncoders are compatible with themselves and EncoderComposite is compatible with every other ColumnEncoders. MultiColumnEncoders are compatible with every encoder
      Overrides:
      mergeAt in class ColumnEncoder
      Parameters:
      other - the encoder that should be merged in

    • updateAllDCEncoders

      public void updateAllDCEncoders()

    • addEncoder

      public void addEncoder(ColumnEncoder other)

    • updateIndexRanges

      public void updateIndexRanges(long[] beginDims, long[] endDims, int colOffset)
      Description copied from class: ColumnEncoder
      Update index-ranges to after encoding. Note that only Dummycoding changes the ranges.
      Specified by:
      updateIndexRanges in interface Encoder
      Overrides:
      updateIndexRanges in class ColumnEncoder
      Parameters:
      beginDims - begin dimensions of range
      endDims - end dimensions of range
      colOffset - is applied to begin and endDims

    • allocateMetaData

      public void allocateMetaData(FrameBlock meta)
      Description copied from interface: Encoder
      Pre-allocate a FrameBlock for metadata collection.
      Parameters:
      meta - frame block

    • getMetaData

      public FrameBlock getMetaData(FrameBlock out)
      Description copied from interface: Encoder
      Construct a frame block out of the transform meta data.
      Parameters:
      out - output frame block
      Returns:
      output frame block?

    • initMetaData

      public void initMetaData(FrameBlock out)
      Description copied from interface: Encoder
      Sets up the required meta data for a subsequent call to apply.
      Parameters:
      out - frame block

    • initEmbeddings

      public void initEmbeddings(MatrixBlock embeddings)
      Overrides:
      initEmbeddings in class ColumnEncoder

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • writeExternal

      public void writeExternal(ObjectOutput out) throws IOException
      Description copied from class: ColumnEncoder
      Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.
      Specified by:
      writeExternal in interface Externalizable
      Overrides:
      writeExternal in class ColumnEncoder
      Parameters:
      out - object output
      Throws:
      IOException - if IOException occurs

    • readExternal

      public void readExternal(ObjectInput in) throws IOException
      Description copied from class: ColumnEncoder
      Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.
      Specified by:
      readExternal in interface Externalizable
      Overrides:
      readExternal in class ColumnEncoder
      Parameters:
      in - object input
      Throws:
      IOException - if IOException occur

    • hasEncoder

      public <T extends ColumnEncoder> boolean hasEncoder(Class<T> type)

    • hasBuild

      public <T extends ColumnEncoder> boolean hasBuild()

    • computeMapSizeEstimate

      public void computeMapSizeEstimate(CacheBlock<?> in, int[] sampleIndices)
      Overrides:
      computeMapSizeEstimate in class ColumnEncoder

    • getAvgEntrySize

      public long getAvgEntrySize()

    • setNumPartitions

      public void setNumPartitions(int nBuild, int nApply)

    • shiftCol

      public void shiftCol(int columnOffset)
      Overrides:
      shiftCol in class ColumnEncoder

    • getDomainSize

      public int getDomainSize()
      Overrides:
      getDomainSize in class ColumnEncoder

    • isRecodeToDummy

      public boolean isRecodeToDummy()

    • isRecode

      public boolean isRecode()

    • isPassThrough

      public boolean isPassThrough()

    • isBin

      public boolean isBin()

    • isBinToDummy

      public boolean isBinToDummy()

    • isHash

      public boolean isHash()

    • isHashToDummy

      public boolean isHashToDummy()