Skip to content

Support for combining repetition_penalty, presence_penalty #274

New issue
New issue
Closed
Closed
Support for combining repetition_penalty, presence_penalty#274
Assignees
byshiuencomly-nvidiajuney-nvidia
Labels
Decoding<NV>Token sampling algorithms in TRTLLM for text gen (top-k, top-p, beam).feature requestNew feature or request. This includes new model, dtype, functionality supporttriagedIssue has been triaged by maintainers
@richardliaw

Description

@richardliaw
richardliaw
opened on Nov 3, 2023

Currently support for these parameters are mutually exclusive. This makes it hard to create an OpenAI compatible API.

cc @akshay-anyscale @Yard1

Metadata

Metadata

Assignees

Labels

Decoding<NV>Token sampling algorithms in TRTLLM for text gen (top-k, top-p, beam).feature requestNew feature or request. This includes new model, dtype, functionality supporttriagedIssue has been triaged by maintainers

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions