Package org.apache.sysds.runtime.instructions.gpu.context

Class GPUMemoryManager

java.lang.Object

org.apache.sysds.runtime.instructions.gpu.context.GPUMemoryManager

public class GPUMemoryManager
extends Object

- All cudaFree and cudaMalloc in SystemDS should go through this class to avoid OOM or incorrect results. - This class can be refactored in future to accept a chunk of memory ahead of time rather than while execution. This will only thow memory-related errors during startup.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	GPUMemoryManager.EvictionPolicyBasedComparator	Class that governs the eviction policy

Constructor Summary

Constructors

Constructor	Description
GPUMemoryManager(GPUContext gpuCtx)

Method Summary

Modifier and Type	Method	Description
void	clearMemory()	Clear the allocated GPU objects
void	clearTemporaryMemory()	Clears up the memory used by non-dirty pointers that are not inside lineage cache
void	free(String opcode, jcuda.Pointer toFree, boolean eager)	Deallocate the pointer
GPULazyCudaFreeMemoryManager	getGPULazyCudaFreeMemoryManager()
GPUMatrixMemoryManager	getGPUMatrixMemoryManager()
long	getSizeAllocatedGPUPointer(jcuda.Pointer ptr)	Get size of allocated GPU Pointer
void	guardedCudaFree(jcuda.Pointer toFree)
void	guardedCudaFree(jcuda.Pointer toFree, boolean noStats)	Note: This method should not be called from an iterator as it removes entries from allocatedGPUPointers and rmvarGPUPointers
jcuda.Pointer	malloc(String opcode, long size, boolean initialize)	Allocate pointer of the given size in bytes.
void	removeGPUObject(GPUObject gpuObj)	Removes the GPU object from the memory manager
String	toString()	Print debugging information

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

GPUMemoryManager

public GPUMemoryManager(GPUContext gpuCtx)

Method Details

getGPUMatrixMemoryManager

public GPUMatrixMemoryManager getGPUMatrixMemoryManager()

getGPULazyCudaFreeMemoryManager

public GPULazyCudaFreeMemoryManager getGPULazyCudaFreeMemoryManager()

getSizeAllocatedGPUPointer

public long getSizeAllocatedGPUPointer(jcuda.Pointer ptr)

Get size of allocated GPU Pointer

Parameters:

ptr - pointer to get size of

Returns:

either the size or -1 if no such pointer exists

malloc

public jcuda.Pointer malloc(String opcode,
 long size,
 boolean initialize)

Allocate pointer of the given size in bytes.

Parameters:

opcode - instruction name

size - size in bytes

initialize - if cudaMemset() should be called

Returns:

allocated pointer

guardedCudaFree

public void guardedCudaFree(jcuda.Pointer toFree,
 boolean noStats)

Note: This method should not be called from an iterator as it removes entries from allocatedGPUPointers and rmvarGPUPointers

Parameters:

toFree - pointer to call cudaFree method on

noStats - do not collect statistics

guardedCudaFree

public void guardedCudaFree(jcuda.Pointer toFree)

free

public void free(String opcode,
 jcuda.Pointer toFree,
 boolean eager)
          throws DMLRuntimeException

Deallocate the pointer

Parameters:

opcode - instruction name

toFree - pointer to free

eager - whether to deallocate eagerly

Throws:

DMLRuntimeException - if error occurs

removeGPUObject

public void removeGPUObject(GPUObject gpuObj)

Removes the GPU object from the memory manager

Parameters:

gpuObj - the handle to the GPU object

clearMemory

public void clearMemory()

Clear the allocated GPU objects

clearTemporaryMemory

public void clearTemporaryMemory()

Clears up the memory used by non-dirty pointers that are not inside lineage cache

toString

public String toString()

Print debugging information

Overrides:

toString in class Object