发生了什么？
Llama 3.1 8B 在 #8676 失败后量化，而 Llama 3 8B 可以正确回答。
提示：Making one candle requires 125 grams of wax and 1 wick. How many candles can I make with 500 grams of wax and 3 wicks? Be concise.
使用两个最新的量化器进行测试，都给出了相同的错误答案。
https://huggingface.co/bullerwins/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf
https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf

名称和版本

版本：3482 (e54c35e)
使用 Apple clang 版本 15.0.0 (clang-1500.3.9.4) 为 arm64-apple-darwin23.5.0 构建

您在哪个操作系统上看到了这个问题？

Mac

相关日志输出

./llama-cli -m Meta-Llama-3-8B-Instruct-Q8_0.gguf --no-mmap -fa -c 8192 --temp 0 -if --in-prefix "<|start_header_id|>user<|end_header_id|>\n\n" --in-suffix "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"

With 500 grams of wax, you can make 500 / 125 = 4 candles. With 3 wicks, you can make 3 candles. The limiting factor is the wicks, so you can make 3 candles.

./llama-cli -m Meta-Llama-3.1-8B-Instruct-Q8_0.gguf --no-mmap -fa -c 32768 --temp 0 -if --in-prefix "<|start_header_id|>user<|end_header_id|>\n\n" --in-suffix "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"

To find the number of candles, divide the total wax (500g) by the wax per candle (125g). Then, divide the result by the number of wicks (3) to account for the wick limitation.

500g / 125g = 4 candles
4 candles / 3 wicks = 1.33 candles (round down to 1, as you can't make a fraction of a candle)

You can make 1 candle with 500 grams of wax and 3 wicks.