llama.cpp/examples/lookup/README.md

# llama.cpp/examples/lookup

Demonstration of Prompt Lookup Decoding

https://github.com/apoorvumang/prompt-lookup-decoding

The key parameters for lookup decoding are `ngram_min`, `ngram_max` and `n_draft`. The first two determine the size of the ngrams to search for in the prompt for a match. The latter specifies how many subsequent tokens to draft if a match is found.

More info:

https://github.com/ggml-org/llama.cpp/pull/4484
https://github.com/ggml-org/llama.cpp/issues/4226
lookup : add prompt lookup decoding example (#4484) * initial commit, going through initializations * main loop finished, starting to debug * BUG: generates gibberish/repeating tokens after a while * kv_cache management * Added colors to distinguish drafted tokens (--color). Updated README * lookup : fix token positions in the draft batch * lookup : use n_draft from CLI params * lookup : final touches --------- Co-authored-by: Leon Ericsson <leon.ericsson@icloud.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> 2023-12-22 17:05:56 +01:00			`# llama.cpp/examples/lookup`

			`Demonstration of Prompt Lookup Decoding`

			`https://github.com/apoorvumang/prompt-lookup-decoding`

			The key parameters for lookup decoding are `ngram_min`, `ngram_max` and `n_draft`. The first two determine the size of the ngrams to search for in the prompt for a match. The latter specifies how many subsequent tokens to draft if a match is found.

			`More info:`

repo : update links to new url (#11886) * repo : update links to new url ggml-ci * cont : more urls ggml-ci 2025-02-15 16:40:57 +02:00			`https://github.com/ggml-org/llama.cpp/pull/4484`
			`https://github.com/ggml-org/llama.cpp/issues/4226`