YD/T 4515-2023
Mobile intelligent terminal intelligent performance benchmark evaluation method (English Version)

Standard No.

YD/T 4515-2023

Language

Chinese, Available in English version

Release Date

2023

Published By

Professional Standard - Post and Telecommunication  CN  /  YD

Latest

YD/T 4515-2023

Introduction

Analysis of Standard Core Framework

Test Level	Core Components	Technical Requirements
Neural Network Model	Inception v3/ResNet50, etc.	Accuracy ≥ Benchmark Value
Inference Framework	TensorFlow Lite/Paddle Lite	Support CPU/GPU/NPU Heterogeneous Computing
Hardware Layer	AI Acceleration Unit	Power Consumption ≤ 120% of Nominal Value

---

Technical specifications for key test scenarios

1. Image classification test

Using data sets such as ImageNet, the test indicators include:

• TOP1 accuracy: reflects the highest confidence classification accuracy
• Inference latency: average processing time for a single image ≤50ms (flagship model)

2. Face recognition test

Based on the LFW data set, core indicators:

When FAR=0.1%	PR should be ≥98%
Feature extraction time	≤30ms/face

---

Suggestions for standard implementation

Key points for setting up the test environment

1. Use the standard test suite to unify the data preprocessing process
2. Close background processes to keep the test environment clean
3. Record initial power and temperature baseline values

Verification of result validity

Need to do:

• Take the average of three repeated tests
• Comparison of different precision models (FP32/INT8)
• Cross-framework (TensorFlow/PyTorch) consistency check

Technology evolution analysis

This standard establishes a multi-dimensional evaluation system for mobile AI performance for the first time. Compared with traditional benchmarking solutions:

Dimensions	Traditional solutions	This standard
Evaluation object	Single hardware	Full-stack capabilities on the end side
Indicator design	Peak computing power	Actual scenario performance

YD/T 4515-2023 history

• 2023 YD/T 4515-2023 Mobile intelligent terminal intelligent performance benchmark evaluation method

Topics on standards and specifications

Standard and Specification