Sips'@slrpnk.net to Selfhosted@lemmy.worldEnglish · 1 year agoCan't relate at all.slrpnk.netexternal-linkmessage-square209linkfedilinkarrow-up11.05Karrow-down124
arrow-up11.03Karrow-down1external-linkCan't relate at all.slrpnk.netSips'@slrpnk.net to Selfhosted@lemmy.worldEnglish · 1 year agomessage-square209linkfedilink
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up3·1 year agoTry a new quantization as well! Like an IQ4-M depending on the size of your GPU, or even better, an 4.5bpw exl2 if you can manage to set up TabbyAPI.
Try a new quantization as well! Like an IQ4-M depending on the size of your GPU, or even better, an 4.5bpw exl2 if you can manage to set up TabbyAPI.