Integrate ChatGPT

[Zabuzard]

Motivation

We want to integrate OpenAI's ChatGPT into TJ-Bot for various things, for example:

• as slash command for users /chatgpt ...
• to identify questions asked in #chit_chat (to then tell them to move to #questions instead)
• to format code (as substitute for our FormatCodeAction)

and possibly more.

Overview

The scope for this feature is to do the first step and add a proof-of-concept into TJ-Bot. That includes:

• API integration itself (either via an existing Java library or by using the REST API directly)
• a simple /chatgpt slash command to try it out
• config entry for the API key
• feature has to be able to disable itself if:
  • API key is invalid/garbage/empty
  • API responds with errors (for example bc we exceeded the quota)

We already created an account on https://platform.openai.com/ and have an API key for development/testing purposes - which will be made available to whoever implements this.

Model

We want the ChatGPT integration to use a simple and cheap model. For example:

gpt-3.5-turbo | $0.002 / 1K tokens

Using the chat-completion model.

Links

It seems that there is an official Java API: https://github.com/TheoKanning/openai-java

Relevant classes would be here: https://github.com/TheoKanning/openai-java/tree/main/api/src/main/java/com/theokanning/openai/completion/chat

Seems to look something like:

ChatCompletionRequest chatCompletionRequest = ChatCompletionRequest.builder()
  .model("gpt-3.5-turbo")
  .messages(chatMessages)
  .frequencyPenalty(.5)
  .temperature(.7)
  .maxTokens(4000)
  .build();

[surajkumar]

Things to consider:

• API availability, Doppey mentioned within the server that the API faces frequent downtime. We should keep existing features (that we intend to switch) as a fallback mechanism. We should also include a fallback mechanism for any new features that we might become reliant on. More so, in the event we run out of credits.
• Price to usage: Detecting questions asked in #chit-chat will be expensive. We should do our own internal checks or create some sophisticated predicate that can determine if something could be a question.
• Flexibilty of the inital system: We want to be able to switch off ChatGPT, change the model and everything else without needing downtime or re-building of the bot. Potentially even the ability to disable certain features.

I'm sure there's more but for now, the first point should be the biggest thing to think about.

Wazei

[Zabuzard]

Good stuff. All highly relevant. In terms of chit-chat pricing, I computed it quickly and I estimate the cost at around 1€ pm (+- 50cents) if using it on all posts in chit-chat. Affordable.

But yeah, for the start I am more aiming for features that would trigger less often, such as code formatting or assisted help (auto-post a chatgpt answer on each question in in #questions) or similar stuff.

But for this first step, its enough if we add a simple slash command that is disabled implicitly by borking the token in the config (bad key, empty key, ...). That way we can react and immediately shut the feature down in case necessary. Then it might be necessary to add a few restrictions to the slash command, such as restricting the message length (200 chars?) and usage per user (20x per day?). Just to ensure nobody can exploit it.

[tmcdonnell2]

May I be assigned to this issue? At least to the initial setup and testing part.

[surajkumar]

May I be assigned to this issue? At least to the initial setup and testing part.

Yes, feel free. I don't think anybody else has made a start. What's your disc name?

[tmcdonnell2]

May I be assigned to this issue? At least to the initial setup and testing part.

Yes, feel free. I don't think anybody else has made a start. What's your disc name?

Loimz

[Zabuzard]

Perfect, enjoy it! Feel free to ask if anything is unclear 😄