Lesson 12 From opinion extraction to private deployment: connect the whole delivery chain
Split opinions before matching labels
In this automotive case, one feedback item may contain component issues, service opinions, and speech-transcription errors. First split it into opinions and extract vehicle, context, sentiment, intent, and text locations. Then select candidate labels from a maintained catalog, review ambiguity, and standardize brand and vehicle codes.
For the fictional feedback “The infotainment system froze again, but after-sales called back quickly,” retain two opinions and their text locations. Check the split before checking classification. Vector retrieval and reranking can help select labels while separating a changing catalog from model interpretation. Decide whether to use this workflow from catalog size, ambiguity, and evaluation results.
Establish a repeatable baseline before considering fine-tuning
Start with a small authorized, anonymized, human-reviewed sample and a repeatable baseline, then compare improvements such as example prompts or fine-tuning. Record how advertising, duplicates, personal information, transcription errors, long-text splitting, and invalid output formats are handled. Repairing a format does not make guessed fields true.
Separate development examples from final evaluation samples; retain later-period data to check changes. Compare every update on the same evaluation set, separately checking extraction, labels, source locations, and human-review volume. Consider fine-tuning after identifying specific errors and resource limits; it is not an obligatory stage.
Agree the interface so the product can use the result
Even when the model service and business platform use different technology stacks, agree on single and batch inputs, output fields, timeouts, and failures. Specify how records trace back to feedback text, how missing fields and duplicate requests are handled, which results need review, and how label versions are recorded. A good score does not replace shared interface and business definitions.
Seven checks for private delivery
| Check | Evidence to retain |
|---|---|
| Environment and offline preparation | Devices, GPU memory, operating system, network limits, authorized material inventory |
| Version compatibility | Matching driver, runtime, framework, or image versions |
| Capacity | Budget for model, runtime overhead, cache, and other services |
| Readiness | Checks for model, retrieval, and business entry points |
| Load testing | Latency, throughput, errors, and queues under a consistent input distribution |
| Reliability | Failure tracing, bounded retries, alerts, stop and recovery exercises |
| Data integration | Duplicate and missing processing, replay, label updates, and write-back checks |
Choose queues, schedulers, and warehouse tools around the customer’s existing systems; agree processing semantics first.
Check memory and version compatibility separately
Budget for model weights, runtime overhead, and the KV cache. Text length and concurrency affect capacity and throughput. Validate multi-GPU partitioning or quantization on the target hardware. With vLLM, check that prebuilt packages are compatible with the CUDA and PyTorch builds. Choose versions and settings for the target environment.
Try this today
Manually split ten fictional comments, retaining source locations and label versions. Simulate one label rename and one service timeout. No model or GPU purchase is required.
Outputs and acceptance
- An “input → opinions → candidate labels → review → write-back” flow table.
- Fixed evaluation examples and two version comparisons, separating data, definition, and model errors.
- An environment checklist that distinguishes known, unknown, and untested items. Do not invent measured latency or capacity.
学完后记得保存进度
Completion marks stay in this browser. Sign in to remember your last chapter and reading position.