Skip to main content

AI Refactoring: Task Checklist

Phase 1: Dependencies & Setup​

  • Install ai, @ai-sdk/openai-compatible, @ai-sdk/react in apps/web/ 
    pnpm add ai @ai-sdk/openai-compatible @ai-sdk/react --filter ever-works-web
  • Verify version alignment with packages/plugins/agent-pipeline/ (ai@^6.0.85)
  • Run pnpm install from root to ensure lockfile is updated

Phase 2: Enable Tool Calling in AiOperations​

2.1 Update Plugin Contracts​

  • Add toolCallId?: string field to ChatMessage interface
    • File: packages/plugin/src/contracts/capabilities/ai-provider.interface.ts
  • Make createStreamingChatCompletion required (not optional) in IAiProviderPlugin
  • Verify ToolDefinition, ToolCall, ChatCompletionOptions.tools already exist (they do)

2.2 Update AiOperations Class​

  • Import ToolMessage, BaseMessage from @langchain/core/messages
    • File: packages/plugin/src/ai/ai-operations.ts
  • Update toLangChainMessages() to handle:
    • role: 'tool' -> ToolMessage with tool_call_id
    • role: 'assistant' with toolCalls -> AIMessage with tool_calls
  • Add bindTools() private method using LangChain's bindTools API:
    • If options.tools present, call llm.bindTools(tools, { tool_choice })
    • Add 'tools' to parameter rejection/retry logic in parseRejectedParam()
  • Update createChatCompletion() to parse tool calls:
    • Extract response.tool_calls from AIMessage
    • Map to ChatCompletionResponse.choices[].message.toolCalls
    • Set finishReason: 'tool_calls' when tool calls present
  • Update createStreamingChatCompletion() to parse tool call chunks:
    • Extract chunk.tool_call_chunks from AIMessageChunk
    • Map to delta.toolCalls in chunk format
    • Handle finishReason: 'tool_calls'
  • Update parseRejectedParam() to handle 'tools' rejection

2.3 Additional Changes​

  • Remove optional streaming guard from AiFacadeService.createStreamingChatCompletion()
    • File: packages/agent/src/facades/ai.facade.ts

2.4 Verify​

  • Run cd packages/plugin && pnpm type-check — passes
  • Run type-check on all 8 AI provider plugins (openai, anthropic, groq, google, mistral, openrouter, ollama, vercel-ai-gateway) — all pass
  • Run cd packages/agent && pnpm test — 33 suites, 892 tests pass
  • Note: langfuse-plugin has pre-existing type-check failure (unrelated to our changes)

Phase 3: OpenAI-Compatible NestJS Endpoint​

3.1 Create DTOs​

  • Create apps/api/src/ai-conversation/dto/openai-compat.dto.ts
    • OpenAiChatCompletionRequestDto with class-validator decorators
    • OpenAiMessageDto, OpenAiToolDefinitionDto, OpenAiFunctionDto classes
    • OpenAiChatCompletionResponse interface (non-streaming)
    • OpenAiChatCompletionChunkResponse interface (streaming)
    • OpenAiToolCallResponse interface

3.2 Create Service​

  • Create apps/api/src/ai-conversation/openai-compat.service.ts
    • Constructor: inject AiFacadeService, WorkRepository
    • handleCompletion(dto, facadeOptions) - non-streaming path
    • handleStreamingCompletion(dto, facadeOptions, res) - SSE streaming path
    • mapToInternalMessages(messages) - OpenAI -> internal ChatMessage format (with tool_calls + tool_call_id)
    • mapToInternalOptions(dto) - Full request mapping (snake_case -> camelCase)
    • mapToOpenAiResponse(response) - Internal -> OpenAI response format (camelCase -> snake_case)
    • mapToOpenAiStreamChunk(chunk, toolCallBaseIndex) - Internal -> OpenAI SSE chunk format with indexed tool_calls
    • resolveWorkContext(options) - Reuses same logic as old service

3.3 Create Controller​

  • Create apps/api/src/ai-conversation/openai-compat.controller.ts
    • POST /api/v1/chat/completions
    • JWT auth via @CurrentUser()
    • Read X-Provider-Override and X-Work-Id headers
    • Route to streaming (SSE) vs non-streaming (JSON) based on body.stream
    • Proper SSE headers: text/event-stream, no-cache, keep-alive, X-Accel-Buffering: no

3.4 Clean Up Old Code​

  • Deleted apps/api/src/ai-conversation/ai-conversation.controller.ts (old NDJSON controller)
  • Deleted apps/api/src/ai-conversation/ai-conversation.service.ts (old streaming service)
  • Updated ai-conversation.module.ts — only registers OpenAiCompatController + OpenAiCompatService
  • Fixed ai.facade.spec.ts — added createStreamingChatCompletion to mock (required after interface change)

3.5 Verify​

  • npx turbo build --filter=ever-works-api — 0 TSC issues, all 6 tasks pass
  • cd packages/agent && pnpm test — 33 suites, 892 tests pass
  • Manual curl testing (deferred to integration testing)

Phase 4: Custom Vercel AI SDK Provider + Next.js Route Handler​

Provider Selection Requirements​

  • X-Provider-Override is always passed — user always has an active AI provider
  • Default: openrouter (auto-selected via resolveEffectiveDefault() in ChatProvider)
  • Auth: encrypted JWT cookie → getAuthAccessCookie() → Bearer token to backend

4.1 Custom Provider​

  • Create apps/web/src/lib/ai/provider.ts
    • createBackendProvider(options) function
    • Uses createOpenAICompatible from @ai-sdk/openai-compatible
    • providerOverride is required (not optional) — always passed as X-Provider-Override header
    • apiKey = JWT token (sent as Authorization: Bearer automatically)
    • baseURL = ${API_URL}/v1 (API_URL already includes /api suffix)
    • Marked server-only — no client-side usage
  • Create apps/web/src/lib/ai/index.ts barrel export

4.2 Next.js Route Handler​

  • Create apps/web/src/app/api/chat/route.ts
    • POST handler with maxDuration = 60
    • Auth: getAuthAccessCookie() + refreshAccessToken() fallback (same as serverFetch)
    • Parse body: extract messages (UIMessage[]), providerOverride (required), workId
    • Returns 400 if providerOverride missing
    • Returns 401 if no auth token after refresh attempt
    • Creates provider with createBackendProvider() pointing to ${API_URL}/v1
    • Calls streamText() with provider('default') + convertToModelMessages(messages)
    • Returns result.toUIMessageStreamResponse()
  • Test end-to-end: web -> Next.js route -> NestJS -> plugin -> LangChain

Phase 6: Frontend Chat UI Refactor​

6.1 Update ChatProvider​

  • Rewrite apps/web/src/components/ai/ChatProvider.tsx
    • Replace useChatHistory with useChat from @ai-sdk/react
    • Configure useChat with DefaultChatTransport({ api: '/api/chat' })
    • Pass providerOverride via body option (always required, defaults to 'openrouter')
    • Set messages with welcome message as initial
    • Expose sendMessage, setMessages, status, error, stop, regenerate through context
    • Add resetChat() method
    • Keep provider fetching logic (unchanged)

6.2 Split into Clean Components​

  • Create apps/web/src/components/ai/ChatMessage.tsx — Single message rendering (user/assistant)
  • Create apps/web/src/components/ai/ChatMessageContent.tsx — Message parts renderer (text, tool calls, streaming dots)
  • Create apps/web/src/components/ai/ChatMessageEdit.tsx — Inline editing textarea with save/cancel
  • Create apps/web/src/components/ai/ChatProviderSelector.tsx — Provider selection pill buttons

6.3 Rewrite ChatInterface​

  • Rewrite apps/web/src/components/ai/ChatInterface.tsx
    • Uses split sub-components (ChatMessage, ChatProviderSelector)
    • Uses useChatContext() for all state
    • isStreaming = status === 'streaming' || status === 'submitted'
    • Form submit calls sendMessage(text) from context
    • Message editing: setMessages to truncate + regenerate() (v6 API, not reload)
    • Reset calls resetChat()
    • Error from error.message
    • Removed: pendingMessageRef, updatePendingMessage, clearPending, all manual streaming

6.4 Refactor Sidebar — Chat as Sidebar-of-Sidebar​

  • Rewrite apps/web/src/components/dashboard/DashboardSidebar.tsx
    • Removed menu/chat mode toggle (no more activeMode state)
    • Sidebar always shows navigation — AI Chat button is a nav item
    • Chat slides out as a secondary panel (positioned at left: sidebarWidth)
    • Panel has its own close button, drag-to-resize handle, and backdrop
    • Works with both collapsed and expanded sidebar states
    • Removed LayoutList import (unused after mode toggle removal)

6.5 Verify​

  • npx turbo build --filter=ever-works-web — build passes, 0 TypeScript errors
  • Old /api/ai-conversations/chat/stream route removed from build output
  • New /api/chat route present in build output

Phase 7: Cleanup​

7.1 Remove Old Files​

  • Deleted apps/web/src/lib/hooks/use-ai-stream.ts
  • Deleted apps/web/src/lib/hooks/use-chat-history.ts
  • Deleted apps/web/src/lib/api/ai-conversation.ts
  • Deleted apps/web/src/app/api/ai-conversations/ (entire work tree)
  • Deleted apps/web/src/lib/utils/next-api.ts
  • Deleted apps/api/src/ai-conversation/ai-conversation.controller.ts (old NDJSON controller)
  • Deleted apps/api/src/ai-conversation/ai-conversation.service.ts (old streaming service)

7.2 Update References​

  • Removed export * from './ai-conversation' from apps/web/src/lib/api/index.ts
  • Removed API_AI_CONVERSATIONS_CHAT_STREAM from constants, added API_CHAT
  • Removed re-exports of deleted types from apps/web/src/lib/api/types-only.ts

7.3 Verify No Broken Imports​

  • Search for use-ai-stream -> 0 results
  • Search for use-chat-history -> 0 results
  • Search for aiConversationAPI -> 0 results
  • Search for nextApiResponseStreaming -> 0 results
  • Search for API_AI_CONVERSATIONS_CHAT_STREAM -> 0 results

7.4 Final Verification​

  • npx turbo build --filter=ever-works-web passes
  • npx turbo build --filter=ever-works-api passes
  • cd packages/agent && pnpm test — 892 tests pass
  • Manual E2E test of chat feature (requires running servers)
  • Manual test of work generation (verify pipelines still work)