| 盛杰,黄鑫鑫,顾艳,刘星星,马双双,徐云峰.AI生成科技论文摘要的效能、缺陷与治理路径.编辑学报,2026,38(3):330-340 |
| AI生成科技论文摘要的效能、缺陷与治理路径 |
| Efficacy,limitations,and governance pathways for AI-generated sci-tech paper abstracts |
| |
| DOI：10.16811/j.cnki.1001-4314.2026.03.015 |
| 中文关键词: 科技期刊 人工智能 生成摘要 多维评估 应对策略 |
| 英文关键词: sci-tech journals artificial intelligence generated abstracts multidimensional evaluation mitigation strategy |
| 基金项目:*江苏省社科应用研究精品工程课题(25SQB-013,25SQC-017)；江苏期刊出版研究课题(2024JSQKB45)；中国农业期刊网研究基金(CAJW2024-042) |
| 作者 | 单位 | | 盛杰 | 江苏大学杂志社,江苏镇江,212013 | | 黄鑫鑫 | 江苏大学杂志社,江苏镇江,212013 | | 顾艳 | 江苏大学杂志社,江苏镇江,212013 | | 刘星星 | 江苏大学杂志社,江苏镇江,212013 | | 马双双 | 江苏大学杂志社,江苏镇江,212013 | | 徐云峰 | 江苏大学杂志社,江苏镇江,212013 |
|
| 摘要点击次数: 172 |
| 全文下载次数: 25 |
| 中文摘要: |
| 为了探究人工智能在科技论文摘要生成中的实际效能,评估不同模型生成摘要的质量差异与能力边界,选取DeepSeek、Kimi、智谱清言、ChatGPT、Claude共5种模型生成摘要,并设置人工撰写组与原摘要组作为对照。采用盲评方式,由30位专家从“内容完整性”、“创新突出性”、“信息密度”与“语言表达”4个维度进行评分,共获取2 100个评价样本的标准化分数与768条专家评语,开展定量与定性相结合的内容分析。定量分析结果显示,原摘要标准化总分显著高于其他各组,Claude模型组综合得分排名第2。方差分析进一步揭示各组在4个维度上的得分存在显著差异。定性分析则发现,AI生成摘要普遍存在“创新点不突出”与“信息冗余”2大核心缺陷,并呈现明显“AI痕迹”,具体表现为“冗余膨胀、模板化、信息幻觉、过度泛化、关键信息缺失、语义漂移”等6类典型问题。定量与定性结果的高度一致性,验证了本研究评价体系的可靠性。研究表明,AI摘要质量呈现显著的模型差异与生成质量问题双重特征,需采取精细化管理策略。建议作者在使用AI工具时主动声明辅助工具类型,优化提示指令模板；编辑部应建立AI摘要专项审查流程,开发涵盖“数据一致性、机制完整性、表述客观性、信息精确性、结论严谨性、表达原创性”6大维度的核查清单,并明确作者对学术内容负最终责任。本研究构建了多维度摘要质量评价标准,系统揭示了AI生成摘要的质量缺陷模式,提出了规范化的辅助摘要生成应用框架,为探索学科适配的人机协同模式、构建兼顾效率与质量的学术出版治理体系提供了实证支撑。 |
| 英文摘要: |
| To examine the practical effectiveness of artificial intelligence in generating sci-tech paper abstracts and to evaluate the quality differences and capability limits of abstracts produced by different models,five AI models—DeepSeek,Kimi,Zhipu Qingyan,ChatGPT,and Claude—were selected for abstract generation. A manually written group and the original abstract group were also set up as controls. A blind review approach was adopted,in which 30 experts rated the abstracts from four dimensions: content completeness,prominence of innovation,information density,and language expression. A total of 2,100 standardized evaluation scores and 768 expert comments were collected,and a combined quantitative and qualitative content analysis was performed. Quantitative analysis shows that the standardized total scores of the original abstracts are significantly higher than those of all other groups,with the Claude model group ranking second in overall score. Analysis of variance further reveals significant differences among the groups across the four evaluation dimensions. Qualitative analysis indicates that AI-generated abstracts generally suffer from two core shortcomings:“lack of prominent innovation”and“information redundancy”,and display noticeable“AI traces”,which manifest in six typical issues:redundant elaboration,templated phrasing,information hallucination,overgeneralization,omission of key information,and semantic drift. The high consistency between quantitative and qualitative results confirms the reliability and validity of the evaluation framework used in this study. It is demonstrated that the quality of AI-generated abstracts is characterized by significant variability across models and inherent generative flaws,calling for refined management strategies. Authors are advised to explicitly declare the use of AI tools and optimize their instruction prompts. Editorial offices are recommended to establish dedicated review procedures for AI-generated abstracts and develop a six-dimension verification checklist covering data consistency,mechanism integrity,objectivity of expression,information accuracy,conclusion rigor,and phrasing originality. Furthermore,authors are expected to bear ultimate responsibility for the academic content. In this study,a multidimensional evaluation standard for abstract quality was constructed,the defect patterns of AI-generated abstracts were systematically revealed,and a standardized application framework for AI-assisted abstract generation was proposed. These contributions provide empirical support for exploring discipline-adapted human-AI collaboration models and for developing an academic publishing governance system that balances efficiency with quality. |
|
查看全文
查看/发表评论 下载PDF阅读器 |
| 关闭 |
|
|
|
|
|
