From the Internet to big data , Has entered a new era , As big data gets stronger , Big data analysis has become an indispensable part of big data technology .
at present , There is a big gap for data analysis talents in the market , The survey shows that , near 60% The company has set up data analysis department , exceed 1/3 Big data analysis has been applied to the daily operation and sales of enterprises .
that , What is big data analysis ？ in general , Big data analysis can be roughly divided into the following five aspects ：
The most basic requirement for big data analysis is visual analysis , Because visual analysis can intuitively present the characteristics of big data .
Data mining algorithms
The theoretical core of big data analysis is data mining algorithm , All kinds of data mining algorithms based on different data types and formats can show the characteristics of data itself more scientifically .
in addition , It's also because of these data mining algorithms , In order to process big data more efficiently .
Predictive analysis capability
One of the areas in which big data analysis will eventually land is predictive analysis , Mining its characteristics from big data , Through scientific modeling , You can bring in new data through the model , To predict future data .
Big data analysis is widely used in network data mining , From the user's search keywords 、 Tag keywords or other input semantics , analysis 、 Judge user needs , In order to better meet the needs of users and match the corresponding advertising .
Data quality and data management
Big data analysis is inseparable from data quality and data management , High quality data and effective data management , Whether in academic research or in the field of commercial application , Can guarantee the authenticity of the analysis results and data value .
Although big data analysis can effectively improve data utilization , however , Like most top-level technologies , Big data analysis also has problems that are not easy to solve .
Data storage problems
With the development of Technology , Data volume from TB Rise to PB,EB Magnitude , If you still use the traditional data storage method , It will certainly cause a lot of inconvenience to big data analysis , This requires the use of dynamic data processing technology , That is, with the regular change of data and display requirements , Irregular processing of data .
meanwhile , A large amount of data can not be directly stored in traditional structured database .
Analysis of resource scheduling problem
The time point and amount of data generated by big data are difficult to calculate , This is a big feature of big data —— uncertainty .
So we need to establish a dynamic response mechanism , Calculation of finite 、 Reasonable allocation and scheduling of storage resources .
in addition , How to get the best analysis results with the least cost is also a problem to be considered .
Professional analysis tools
While developing data analysis technology , Traditional software tools are no longer applicable , and , At present, human science and technology is not mature , There is still a certain distance from the development of general software that can meet the needs of big data analysis .