How can I stream token-by-token output from OpenAI's chat completions API using PHP and JavaScript?

Below is an explanation of the process along with an example using streaming mode. The key is to have the API send you the response incrementally (token-by-token) by enabling streaming on your API call. Once you receive the streamed response, you can update your UI as each token arrives.

How to Enable Streaming

When you create your chat completion request, include "stream": true in your JSON body. This tells the OpenAI API to return tokens as soon as they're generated instead of waiting for the full completion.

Example Implementation

If you were to use Node.js or TypeScript, the streaming can be implemented like this:

JAVASCRIPTCopied
import OpenAI from "openai";

const openai = new OpenAI();

async function main() {
  const stream = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Say this is a test" }],
    stream: true,
  });
  for await (const chunk of stream) {
    // Update your UI or output token-by-token.
    process.stdout.write(chunk.choices[0]?.delta?.content || "");
  }
}

main();

Adapting to Your PHP & JavaScript Setup

Since your tool currently uses PHP to forward the request and then AJAX on the JavaScript side:

1. Update the API Request in PHP:

   • Make sure that when PHP makes the HTTP request, it sets the stream field to true in the JSON payload.
   • Instead of waiting for the full response, your PHP script should read the streamed response as it arrives. This might require using lower-level HTTP client features (such as cURL with streaming enabled).

2. Forwarding the Stream to the Browser:

   • Depending on your architecture, consider one of these approaches:
     • Server-Sent Events (SSE): Let your PHP script act as a proxy that pushes server-sent events to the browser as tokens arrive.
     • WebSocket: Use PHP or another backend language that supports WebSockets to relay the streaming tokens in real time.

3. Update the UI in JavaScript:

   • In your JavaScript, listen for the individual token events and update the display (e.g., appending each token to a text element) as soon as a new token is received.
   • If you go with SSE, you can use the using EventSource:

JAVASCRIPTCopied
const evtSource = new EventSource('/path-to-your-php-proxy');
evtSource.onmessage = (event) => {
  const token = event.data; 
  // Append the token to your output element.
  document.getElementById('output').textContent += token;
};

Summary

• Enable streaming: Set "stream": true in your API request.
• Proxy the stream: Update your PHP script to handle streaming responses and forward them to the client (using SSE or WebSockets).
• Update the UI: Use JavaScript to listen for the streamed tokens and render them token-by-token.

Adapting the provided Node.js example to your PHP/JavaScript stack involves ensuring that each part of your application supports incremental updates. This will provide a smoother, real-time token-by-token rendering experience.

I hope this helps!