change qwen parallelism to 2
This commit is contained in:
@@ -71,7 +71,7 @@ spec:
|
||||
- -c
|
||||
- "131072"
|
||||
- -np
|
||||
- "1"
|
||||
- "2"
|
||||
- --cont-batching
|
||||
- --cache-type-k
|
||||
- q8_0
|
||||
|
||||
Reference in New Issue
Block a user