Are --load-in-8bit and --bf16 mutually exclusive?

[Slug-Cat]

If I add both, the server seems to run fine but I don't know whether they're actually doing anything.

[oobabooga]

Yes, --bf16 is ignored if --load-in-8bit is specified:

text-generation-webui/server.py

Line 139 in 596732a

params.append("load_in_8bit=True" if args.load_in_8bit else "torch_dtype=torch.bfloat16" if args.bf16 else "torch_dtype=torch.float16")

[ack226]

Is one generally better to use than the other, assuming my GPU supports both?