vllm部署qwen3-32b模型，推理服务兼容openai服务API 支持openclaw调用
1. 宿主机配置GPUHG DCU K100_AI -E4x16 64GBOSKylin Linux Advanced Server V10 (Halberd)基于vllm qwen3-32B模型 搭建openclaw2、宿主机启动VLLM docker#启动海光版本vllm docker docker run -it \ --name qwen3-32b \ --device/dev/kfd \ --privileged \ --networkhost \ --device/dev/dri \ --cap-addSYS_PTRACE \ --security-opt seccompunconfined \ -v /opt/hyhal:/opt/hyhal:ro \ -v /public/model/qwen/Qwen3-32B/:/workspace/Qwen3-32B:ro \ --group-add video \ --shm-size 64G \ -w /workspace \ image.sourcefind.cn:5000/dcu/admin/base/vllm:0.9.2-ubuntu22.04-dtk25.04.1-rc5-rocblas104381-0915-das1.6-py3.10-20250916-rc2 \ bash #进入容器 docker exec -it qwen3-32b bash3、容器内部环境变量设置#设置环境变量 model_path/workspace/Qwen3-32B modelqwen3-32B tp4 time$(date %m%d-%H%M) modecudagraph data_typefp16 port8085 #启动服务 VLLM_LOG_LEVELDEBUG nohup vllm serve $model_path \ --served-model-name $model \ --enable-auto-tool-choice \ --tool-call-parser hermes \ --host 0.0.0.0 \ --port $port \ --dtype float16 \ --tensor-parallel-size $tp \ --max-num-seqs 1024 \ --trust-remote-code \ --distributed-executor-backendmp \ --no-enable-prefix-caching \ --max-model-len 40960 \ --max-seq-len-to-capture 40960 \ /workspace/logs/vllm_$(date %Y%m%d_%H%M%S).log 21 4、获取模型列表并请求测试curl http://10.9.90.91:8085/v1/models curl http://10.9.90.91:8085/v1/chat/completions -H Content-Type: application/json -d { model: Qwen3-32B, messages: [ {role: system, content: You are a helpful assistant.}, {role: user, content: 你好请介绍一下自己} ], temperature: 0.7, max_tokens: 1024 }5、获取模型apt update apt install -y git-lfs git lfs install git clone https://www.modelscope.cn/Qwen/Qwen3-32B.git6、简单的python proxy 便于分析客户端请求体#cat proxy/proxy.py import json from flask import Flask, request, Response import requests app Flask(__name__) # vLLM 服务的真实地址 VLLM_URL http://127.0.0.1:8085 app.route(/path:path, methods[GET, POST, PUT, DELETE]) def proxy(path): url f{VLLM_URL}/{path} # 1. 打印请求头和请求体 print(f\n{*50}) print(f收到请求: {request.method} {path}) if request.is_json: # 这里就是你最想看的 JSON 内容 print(请求体 (JSON):) print(json.dumps(request.json, indent2, ensure_asciiFalse)) else: print(f请求体 (Raw): {request.get_data()}) print(f{*50}\n) # 2. 转发请求到真实的 vLLM headers {k: v for k, v in request.headers if k.lower() ! host} try: resp requests.request( methodrequest.method, urlurl, headersheaders, datarequest.get_data(), cookiesrequest.cookies, allow_redirectsFalse, timeout300 ) # 3. 构造并返回响应 excluded_headers [content-encoding, content-length, transfer-encoding, connection] resp_headers [ (name, value) for name, value in resp.raw.headers.items() if name.lower() not in excluded_headers ] return Response(resp.content, resp.status_code, resp_headers) except Exception as e: return fProxy Error: {str(e)}, 500 if __name__ __main__: # 监听 8086 端口 app.run(host0.0.0.0, port8086)
)
