4.4 Data Analysis and Mining在线视频

这节课
In this lecture

将介绍几种数据处理技术的流程
we will move on to section 4.4 the technical architecture of data analysis and mining

数据分析是智能电网大数据处理的核心
Data analysis is the core of smart grid big data processing

数据集成和清洗是数据分析的基础
Data integration and cleaning are the basis of data analysis

大数据的价值产生于数据分析
The value of big data comes from data analysis。

由于智能电网大数据具有海量、复杂多样、变化快等特性，
Because smart grid big data has the characteristics of massive, complex and diverse, fast change

传统的数据分析算法很多已不再适用
many traditional data analysis algorithms are no longer applicable and

需要采用新的技术架构、数据分析方法或对现有数据分析方法进行改进
need to adopt new technology architecture, data analysis methods or improve the existing data analysis methods

智能电网大数据分析挖掘的常用方法包括
The common methods of big data analysis and mining in smart grid include

统计分析方法、数据挖掘方法、机器学习方法及新型方法等
statistical analysis, data mining, machine learning and new methods

统计分析方法
Statistical analysis methods

统计分析方法试通过整理、分析、描述数据等手段
Statistical analysis methods attempt to discover the nature of the object to be measured

发现被测对象本质，甚至预测被测对象未来的一类方法
or even to predict the future of the object to be measured, by means of collating, analyzing, and describing data

统计分析可以分为大型数据集提供两种服务：描述和推断
Statistical analysis can be divided into large data sets to provide two services: description and inference

更复杂的多元统计分析技术有
More complex multivariate statistical analysis techniques include

多重回归分析（简称回归分析）、判别分析、聚类分析
multiple regression analysis (regression analysis for short), discriminant analysis, clustering analysis

主元分析、对应分析、因子分析
main element analysis, correspondence analysis, factor analysis

典型相关分析、多元方差分析等
typical correlation analysis, multivariate analysis of variance, etc

数据挖掘方法
Data mining

传统的数据挖掘是在大型数据存储库中自动地发现有用信息的过程
Traditional data mining is a process of automatically discovering useful information in large data repositories

其方法主要有
the main methods are

分类分析、回归分析、关联分析
classification analysis, regression analysis, association analysis

聚类分析、异常检测和汇总等六种
clustering analysis, anomaly detection, aggregation

2006年，电气和电子工程师协会（IEEE ICDM）
In 2006, the Institute of Electrical and Electronics Engineers (IEEE ICDM)

评选出十个最具影响力的数据挖掘算法
selected the ten most influential data mining algorithms

包括：分析决策树法（C4.5）、K均值（K-means）聚类算法
including: analytical decision tree method (C4.5), K-mean (K-means) clustering algorithm

支持向量机算法（SVM）、布尔关联规则频繁项集算法（Apriori）
support vector machine (SVM) algorithm, Boolean association rule frequent term set algorithm (Apriori)

最大期望算法（EM）、提升算法（AdaBoost）
maximum expectation algorithm (EM), boost algorithm (AdaBoost)

网页排名算法（PageRank）、K近邻算法（kNN）、朴素贝叶斯算法（NB）
page rank algorithm (PageRank), K-nearest neighbor algorithm (kNN), Park Bayesian algorithm (NB)

和分类回归树算法（CART）
and categorical regression tree (CART).Translated with www.DeepL.com/Translator (free version)

机器学习方法
Machine learning methods

机器学习算法是一类从数据中自动分析获得规律
Machine learning algorithms are a class of algorithms that automatically analyze data to obtain laws from it

并利用规律对未知数据进行预测的算法
and use them to make predictions about unknown data

机器学习大体上可分为
Machine learning can be broadly classified into

监督学习无监督学习半监督学习和增强学习等几类
supervised learning, unsupervised learning, semi-supervised learning and reinforcement learning

这个表格告诉我们机器学习的分类，定义和一些典型的数学方法
This table tells us the classification, definitions and some typical mathematical methods of machine learning

对于监督学习而言，从给定的训练数据集中学习出一个函数
For supervised learning, A function is learned from the given training data set

当新的数据到来时，可以根据这个函数预测结果
When new data comes, the result can be predicted according to this function

监督学习的训练集要求是包括输入和输出
The training set of supervised learning must include input and output

也可以说是特征和目标
or features and objectives

训练集中的目标是人为标注的
Objectives in training sets are manually marked

典型的监督学习中分类的数学方法有
Some typical mathematical models for classification algorithms are

K近邻；决策树；支持向量机
K nearest neighbor,decision tree,support vector machine

贝叶斯分类器；集成学习；隐马尔可夫模型等
Bayesian classifier,ensemble learning,hidden Markov model

典型的监督学习中回归数学方法有神经网络和高斯过程回归
Two typical mathematical models for regression algorithms are neural networks,Gaussian process regression

与监督学习相比，监督学习的训练集没有人为标注的结果
Compared with supervised learning, there’s no manually labeled result in training sets

无监督学习又分了聚类和规则学习两个子类
Clustering and rule learning are two typical types of unsupervised learning

常见的聚类模型有
Three most commonly used clustering methods are

自组织映射；层类聚集；聚类分析
self-organizing mapping, hierarchical clustering and cluster analysis

半监督学习介于监督学习和无监督学习之间同时使用有标记和无标记数据
Semi supervised learning is between supervised and unsupervised learning,using both labeled and unlabeled data

常见的学习模型有生成式模型；半监督SVM
Some common learning models are generative model, semi supervised SVM

协同训练；图半监督学习
collaborative training, graph-based semi-supervised

增强学习通过观察来学习做成如何的动作
Reinforcement learning can learn what to do by observing

每个动作都会对环境有所影响
Each action will have an impact on the environment

学习对象根据观察到的周围环境的反馈来做出判断
Learning objects can make judgments based on the feedback of the observed surroundings

常见的学习模型有Q学习、时间差分学习
Common learning models include Q-learning and time difference learning

新兴方法
Latest approaches

随着数据规模的增大
With the increase of data scale

对机器学习算法的可扩展性、稀疏性和鲁棒性也提出了更高要求
the scalability, sparseness and robustness of machine learning algorithm are also required

为从智能电网大数据中获得更准确、更深层次的知识
In order to obtain more accurate and deeper knowledge from the big data of intelligent grid

需要提升分析挖掘系统对数据的认知计算能力
it is necessary to improve the cognitive computing ability of the analytical mining system on the data

采用人工智能方法使
using artificial intelligence methods to make

分析挖掘系统具备对数据的理解、推理、发现和决策能力
the analytical mining system with the ability to understand, reason, discover and make decisions on the data

交互式可视化分析、深度学习、随机矩阵理论、自然语言处理
Interactive visual analysis, deep learning, stochastic matrix theory, natural language processing

群智能等新的数据分析方法
group intelligence and other new data analysis methods

也成为智能电网大数据分析挖掘的重要技术
have also become important techniques for big data of intelligent grid analysis mining

本节课的内容就到这里
Ok, that’s all for this lecture

下节课将介绍数据可视化技术
Next time, we will learn some technical details of data visualization