Package org.apache.sysds.runtime.transform.encode

Class ColumnEncoderRecode

java.lang.Object
org.apache.sysds.runtime.transform.encode.ColumnEncoder
org.apache.sysds.runtime.transform.encode.ColumnEncoderRecode
All Implemented Interfaces:
Externalizable, Serializable, Comparable<ColumnEncoder>, Encoder
public class ColumnEncoderRecode extends ColumnEncoder
See Also:

  • Field Details

    • SORT_RECODE_MAP

      public static boolean SORT_RECODE_MAP

  • Constructor Details

    • ColumnEncoderRecode

      public ColumnEncoderRecode(int colID)

    • ColumnEncoderRecode

      public ColumnEncoderRecode()

  • Method Details

    • constructRecodeMapEntry

      public static String constructRecodeMapEntry(String token, Integer code)
      Returns the Recode map entry which consists of concatenation of code, delimiter and token.
      Parameters:
      token - is part of Recode map
      code - is code for token
      Returns:
      the concatenation of token and code with delimiter in between

    • constructRecodeMapEntry

      public static String constructRecodeMapEntry(Object token, Integer code, StringBuilder sb)

    • splitRecodeMapEntry

      public static String[] splitRecodeMapEntry(String value)
      Splits a Recode map entry into its token and code.
      Parameters:
      value - concatenation of token and code with delimiter in between
      Returns:
      string array of token and code

    • getCPRecodeMaps

      public Map<Object,Integer> getCPRecodeMaps()

    • getCPRecodeMapsPartial

      public HashSet<Object> getCPRecodeMapsPartial()

    • sortCPRecodeMaps

      public void sortCPRecodeMaps()

    • computeMapSizeEstimate

      public void computeMapSizeEstimate(CacheBlock<?> in, int[] sampleIndices)
      Overrides:
      computeMapSizeEstimate in class ColumnEncoder

    • build

      public void build(CacheBlock<?> in)
      Description copied from interface: Encoder
      Build the transform meta data for the given block input. This call modifies and keeps meta data as encoder state.
      Parameters:
      in - input frame block

    • getBuildTask

      public Callable<Object> getBuildTask(CacheBlock<?> in)
      Overrides:
      getBuildTask in class ColumnEncoder

    • getPartialBuildTask

      public Callable<Object> getPartialBuildTask(CacheBlock<?> in, int startRow, int blockSize, HashMap<Integer,Object> ret, int p)
      Overrides:
      getPartialBuildTask in class ColumnEncoder

    • getPartialMergeBuildTask

      public Callable<Object> getPartialMergeBuildTask(HashMap<Integer,?> ret)
      Overrides:
      getPartialMergeBuildTask in class ColumnEncoder

    • prepareBuildPartial

      public void prepareBuildPartial()
      Description copied from class: ColumnEncoder
      Allocates internal data structures for partial build.
      Specified by:
      prepareBuildPartial in interface Encoder
      Overrides:
      prepareBuildPartial in class ColumnEncoder

    • buildPartial

      public void buildPartial(FrameBlock in)
      Description copied from class: ColumnEncoder
      Partial build of internal data structures (e.g., in distributed spark operations).
      Specified by:
      buildPartial in interface Encoder
      Overrides:
      buildPartial in class ColumnEncoder
      Parameters:
      in - input frame block

    • mergeAt

      public void mergeAt(ColumnEncoder other)
      Description copied from class: ColumnEncoder
      Merges another encoder, of a compatible type, in after a certain position. Resizes as necessary. ColumnEncoders are compatible with themselves and EncoderComposite is compatible with every other ColumnEncoders. MultiColumnEncoders are compatible with every encoder
      Overrides:
      mergeAt in class ColumnEncoder
      Parameters:
      other - the encoder that should be merged in

    • getNumDistinctValues

      public int getNumDistinctValues()

    • allocateMetaData

      public void allocateMetaData(FrameBlock meta)
      Description copied from interface: Encoder
      Pre-allocate a FrameBlock for metadata collection.
      Parameters:
      meta - frame block

    • getMetaData

      public FrameBlock getMetaData(FrameBlock meta)
      Description copied from interface: Encoder
      Construct a frame block out of the transform meta data.
      Parameters:
      meta - output frame block
      Returns:
      output frame block?

    • initMetaData

      public void initMetaData(FrameBlock meta)
      Construct the recodemaps from the given input frame for all columns registered for recode.
      Parameters:
      meta - frame block

    • writeExternal

      public void writeExternal(ObjectOutput out) throws IOException
      Description copied from class: ColumnEncoder
      Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.
      Specified by:
      writeExternal in interface Externalizable
      Overrides:
      writeExternal in class ColumnEncoder
      Parameters:
      out - object output
      Throws:
      IOException - if IOException occurs

    • readExternal

      public void readExternal(ObjectInput in) throws IOException
      Description copied from class: ColumnEncoder
      Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.
      Specified by:
      readExternal in interface Externalizable
      Overrides:
      readExternal in class ColumnEncoder
      Parameters:
      in - object input
      Throws:
      IOException - if IOException occur

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • getRcdMap

      public Map<Object,Integer> getRcdMap()

    • toString

      public String toString()
      Overrides:
      toString in class Object