diff --git a/docs/12 - OpenAI API.md b/docs/12 - OpenAI API.md index 120af127..17c83a3d 100644 --- a/docs/12 - OpenAI API.md +++ b/docs/12 - OpenAI API.md @@ -45,7 +45,7 @@ openai-debug: 1 ### Examples -For the documentation with all the parameters, consult `http://127.0.0.1:5000/docs` or the [typing.py](https://github.com/oobabooga/text-generation-webui/blob/main/extensions/openai/typing.py) file. +For the documentation with all the parameters and their types, consult `http://127.0.0.1:5000/docs` or the [typing.py](https://github.com/oobabooga/text-generation-webui/blob/main/extensions/openai/typing.py) file. The official examples in the [OpenAI documentation](https://platform.openai.com/docs/api-reference) should also work, and the same parameters apply (although the API here has more optional parameters). @@ -181,7 +181,7 @@ while True: print() ``` -### Python completions example with streaming +#### Python completions example with streaming Start the script with `python -u` to see the output in real time. @@ -216,8 +216,7 @@ for event in client.events(): print() ``` -### Client Application Setup - +### Third-party application setup You can usually force an application that uses the OpenAI API to connect to the local API by using the following environment variables: @@ -229,18 +228,18 @@ or ```shell OPENAI_API_KEY=sk-111111111111111111111111111111111111111111111111 -OPENAI_API_BASE=http://127.0.0.1:500/v1 +OPENAI_API_BASE=http://127.0.0.1:5000/v1 ``` -With the [official python openai client](https://github.com/openai/openai-python), set the `OPENAI_API_BASE` environment variables: +With the [official python openai client](https://github.com/openai/openai-python), the address can be set like this: ```shell -# Sample .env file: -OPENAI_API_KEY=sk-111111111111111111111111111111111111111111111111 -OPENAI_API_BASE=http://0.0.0.0:5001/v1 -``` +import openai -If needed, replace 127.0.0.1 with the IP/port of your server. +openai.api_key = "..." +openai.api_base = "http://127.0.0.1:5000/v1" +openai.api_version = "2023-05-15" +``` If using .env files to save the `OPENAI_API_BASE` and `OPENAI_API_KEY` variables, make sure the .env file is loaded before the openai module is imported: @@ -284,35 +283,10 @@ In short, the all-MiniLM-L6-v2 model is 5x faster, 5x smaller ram, 2x smaller st Warning: You cannot mix embeddings from different models even if they have the same dimensions. They are not comparable. -### API Documentation & Examples - -The OpenAI API is well documented, you can view the documentation here: https://platform.openai.com/docs/api-reference - -Examples of how to use the Completions API in Python can be found here: https://platform.openai.com/examples -Not all of them will work with all models unfortunately, See the notes on Models for how to get the best results. - -Here is a simple python example. - -```python -import os -os.environ['OPENAI_API_KEY']="sk-111111111111111111111111111111111111111111111111" -os.environ['OPENAI_API_BASE']="http://0.0.0.0:5001/v1" -import openai - -response = openai.ChatCompletion.create( - model="x", - messages = [{ 'role': 'system', 'content': "Answer in a consistent style." }, - {'role': 'user', 'content': "Teach me about patience."}, - {'role': 'assistant', 'content': "The river that carves the deepest valley flows from a modest spring; the grandest symphony originates from a single note; the most intricate tapestry begins with a solitary thread."}, - {'role': 'user', 'content': "Teach me about the ocean."}, - ] -) -text = response['choices'][0]['message']['content'] -print(text) -``` - ### Compatibility & not so compatibility +Note: the table below may be obsolete. + | API endpoint | tested with | notes | | ------------------------- | ---------------------------------- | --------------------------------------------------------------------------- | | /v1/chat/completions | openai.ChatCompletion.create() | Use it with instruction following models | @@ -335,11 +309,12 @@ print(text) | /v1/fine-tunes\* | openai.FineTune.\* | not yet supported | | /v1/search | openai.search, engines.search | not yet supported | - #### Applications Almost everything needs the `OPENAI_API_KEY` and `OPENAI_API_BASE` environment variable set, but there are some exceptions. +Note: the table below may be obsolete. + | Compatibility | Application/Library | Website | Notes | | ------------- | ---------------------- | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | ✅❌ | openai-python (v0.25+) | https://github.com/openai/openai-python | only the endpoints from above are working. OPENAI_API_BASE=http://127.0.0.1:5001/v1 |