Help Regarding memorization

Hello there
I am currently newly using the chatglmv4 model, i need some advice/help regarding the memorization of the chatglm like how i can store the keywords in chatglm effectivly like if i load too much context the gpu vram would be high and if i trim them the history data will be lost is there any effective way to do it , if you need i can share my use case in brief here