# Phase 8: Order Processing (BullMQ + X-API) **Status:** ⏳ Todo **Progress:** 0/15 tasks (0%) **Started:** - **Completed:** - **Assigned to:** - --- ## Overview Implement asynchronous order processing: BullMQ queue for order submission to X-API, worker for processing orders, retry logic, and BullBoard dashboard for monitoring. **Goal:** Orders are reliably submitted to X-API (NAV ERP) after successful payment. --- ## Dependencies - ✅ Phase 2: Database (orders table needed) - ✅ Phase 7: Payment (orders created after payment) - ✅ Docker Redis running (from docker-compose.dev.yml) - ⚠️ **Required:** X-API credentials (USERNAME, PASSWORD, BASE_URL) --- ## Tasks ### BullMQ Setup - [ ] Install BullMQ + ioredis ```bash pnpm add bullmq ioredis pnpm add -D @bull-board/api @bull-board/nuxt ``` - [ ] Configure Redis connection - File: `server/utils/redis.ts` ```typescript import { Redis } from 'ioredis' const config = useRuntimeConfig() export const redis = new Redis({ host: config.redisHost, port: config.redisPort, password: config.redisPassword, maxRetriesPerRequest: null, }) ``` - [ ] Create order queue - File: `server/queues/orderQueue.ts` ```typescript import { Queue } from 'bullmq' import { redis } from '../utils/redis' export const orderQueue = new Queue('x-api-orders', { connection: redis, defaultJobOptions: { attempts: 5, backoff: { type: 'exponential', delay: 2000 }, removeOnComplete: 1000, removeOnFail: false, }, }) ``` - [ ] Create order worker - File: `server/workers/orderWorker.ts` ```typescript import { Worker } from 'bullmq' import { redis } from '../utils/redis' export const orderWorker = new Worker( 'x-api-orders', async (job) => { const { orderId } = job.data // 1. Fetch order from DB with items and user const order = await db.query.orders.findFirst({ where: eq(orders.id, orderId), with: { items: true, user: true }, }) // 2. Transform to X-API format const payload = transformOrderToXAPI(order, order.user) // 3. Submit to X-API const result = await submitOrderToXAPI(payload) // 4. Update order status await db .update(orders) .set({ status: 'completed', xapiResponse: result }) .where(eq(orders.id, orderId)) return result }, { connection: redis, concurrency: 5, limiter: { max: 10, duration: 1000 }, } ) ``` ### X-API Client - [ ] Create X-API client utility - File: `server/utils/xapi/client.ts` - Functions: - `submitOrderToXAPI(payload)` - Submit order with retry logic & Basic Auth - See: [CLAUDE.md: X-API Order Transformation](../CLAUDE.md#x-api-order-transformation-pattern) - [ ] Implement transformOrderToXAPI function - File: `server/utils/xapi/transformer.ts` - Transform order from DB schema to X-API schema - Critical transformations: - Prices: EUR (Decimal) → Cents (Integer): `Math.round(price * 100)` - Dates: JavaScript Date → ISO 8601 UTC: `.toISOString()` - Line numbers: 10000, 20000, 30000... (multiples of 10000) - Salutation: 'male' → 'HERR', 'female' → 'FRAU', other → 'K_ANGABE' - See: [docs/ARCHITECTURE.md: X-API Format](../docs/ARCHITECTURE.md#34-x-api-order-transformation) - [ ] Implement submitOrderToXAPI with retry - Exponential backoff: 1s, 3s, 9s - Max 3 retries - HTTP Basic Auth header - Timeout: 30 seconds - Log all attempts - See: [CLAUDE.md: X-API Pattern](../CLAUDE.md#x-api-order-transformation-pattern) ### API Endpoints - [ ] Create /api/orders/index.post.ts endpoint - Protected (requires auth) - Body: { billingAddress, paymentId } - Create order record in DB - Create order_items from cart - Queue order for X-API submission - Return: { orderId, orderNumber } - [ ] Create /api/orders/[id].get.ts endpoint - Protected (requires auth) - Fetch order by ID (only user's own orders) - Include order items with product details - Return: Order object ### Testing - [ ] Test queue processing - Add job to queue manually: `orderQueue.add('submit-order', { orderId: '...' })` - Verify worker picks up job - Verify job completes successfully - Check BullBoard dashboard - [ ] Test X-API submission (mock) - Create mock X-API endpoint for testing - Submit order via queue - Verify transformation is correct - Verify Basic Auth header is present - [ ] Add error handling & logging - Log all queue events (active, completed, failed, stalled) - Log X-API requests/responses - Handle X-API errors gracefully - Update order status on failure ### Monitoring - [ ] Setup BullBoard dashboard - File: `server/api/admin/queues.ts` ```typescript import { createBullBoard } from '@bull-board/api' import { BullMQAdapter } from '@bull-board/api/bullMQAdapter' import { NuxtAdapter } from '@bull-board/nuxt' const serverAdapter = new NuxtAdapter() serverAdapter.setBasePath('/admin/queues') createBullBoard({ queues: [new BullMQAdapter(orderQueue)], serverAdapter, }) export default fromNodeMiddleware(serverAdapter.registerPlugin()) ``` - Access at: http://localhost:3000/admin/queues - [ ] Test retry logic - Simulate X-API failure (wrong credentials or mock 500 error) - Verify job retries with exponential backoff - Verify job moves to failed after 5 attempts - Check retry logs - [ ] Document order processing - Document queue flow: payment → queue → worker → X-API - Document retry strategy - Document error handling and recovery - Document how to monitor queues --- ## Acceptance Criteria - [x] BullMQ is installed and configured - [x] Redis connection is working - [x] Order queue is created - [x] Order worker processes jobs correctly - [x] transformOrderToXAPI transforms orders correctly - [x] submitOrderToXAPI submits with Basic Auth and retry logic - [x] /api/orders endpoints create orders and queue jobs - [x] Queue processing works end-to-end - [x] X-API submissions succeed (or fail gracefully) - [x] Error handling and logging are comprehensive - [x] BullBoard dashboard is accessible and functional - [x] Retry logic works as expected - [x] Order processing is documented --- ## Notes - **Async Processing:** Orders are queued immediately, processed in background - **Job Timeout:** 60 seconds per job (X-API timeout + overhead) - **Concurrency:** 5 jobs processed simultaneously - **Rate Limit:** 10 requests/second to X-API - **Failed Jobs:** Kept in Redis for manual inspection (not auto-deleted) --- ## Blockers - ⚠️ **X-API Credentials:** Cannot test real submission without credentials - **Workaround:** Use mock X-API endpoint for testing, document real integration --- ## Related Documentation - [docs/ARCHITECTURE.md: Queue Architecture](../docs/ARCHITECTURE.md#35-queue-architektur-bullmq--redis) - [docs/ARCHITECTURE.md: X-API Format](../docs/ARCHITECTURE.md#34-x-api-order-transformation) - [docs/TECH_STACK.md: BullMQ](../docs/TECH_STACK.md#52-queue-system-bullmq) - [CLAUDE.md: X-API Pattern](../CLAUDE.md#x-api-order-transformation-pattern)