Reflections on the era of big data (1)
We are no longer keen on finding causality, but should look for the correlation between things. This proposition is my biggest feeling when I read this book. Personally, it is also the core idea of this book. Let's start from the beginning. First of all, the book puts forward a proposition that subverts my previous cognition-? Not atoms but information is the origin of everything? Seeing the world as a sea of information and understandable data provides us with an unprecedented perspective. This is a world outlook that can penetrate into all fields of life. This idea is described in a paragraph in the last part of this book. I put it at the forefront because I think it is the premise of talking about the digital world, and naturally it is also the premise of talking about big data. There is a section in the middle of the book about the difference between digitalization and digitalization. After sorting out my brain, the proposition of digital world is listed as the second step of big data thinking. Writing here, I can't help but reflect on whether I have understood the essence of the book (the essence I think). This is the first sentence. Because looking back at my whole thinking, I still think according to the old causal thinking mode. Another attraction of this book is that there are many viewpoints, which will be discussed from a philosophical point of view. Although I don't have much ink in my stomach, when I read these descriptions, I will find that I will better understand the proposition put forward by the author. For example, there is a passage in the book
When we say that human beings know the world through causality, we mean that we use two basic methods to understand and explain various phenomena in the world: one is through rapid and illusory causality, and the other is through slow and orderly causality. Big data will change the role of these two basic methods in our understanding of the world.
When attaching some examples, use what the author provided? Essence? At first glance, it's easy to understand that it is. Okay, so what has big data changed us? The author gives three points.
The essence of big data lies in three changes when we analyze information. These changes are about changing the way we understand and form society.
The first change is that in the era of big data, we can analyze more data, and sometimes even process all the data related to a specific phenomenon, instead of relying on random sampling (sample = population).
The second change is that there are too many research data, and we are no longer keen on pursuing accuracy.
The third change is caused by the first two changes, that is, we are no longer keen on finding causality, but should look for the correlation between things. Big data tells us? What is this? Instead of. Why? . In the era of big data, we don't need to know the reasons behind the phenomenon, we just need to let the data speak for itself.
As we all know, the human brain has such a function, it will combine new input stimuli or information with? Past experience or accumulated knowledge? Compare, then adjust and accept. If the new reality in front of you can't coordinate with the inherent information stored in your brain, you will unconsciously refuse to accept the new reality (as if you didn't see it); Or through their own little knowledge of arbitrary speculation, so that their awareness of the situation deviated from reality (illusion). This is an instinct of human beings, aiming at keeping oneself calm.
So the author calls it a revolution.
Having said that, what does big data bring us? Here, I just want to talk about what I feel the most, and I can understand other interests by myself. Of course, there are many books, and the most is how much wealth XXX companies or individuals have created by using big data. Aside from these superficial things, what interests me or scares me most is prediction. This is the core thing brought by big data. There is no need to repeat the reasons for being tempted. The computer will tell you when to buy a two-color ball and win the first prize. Think about whether you are a little excited. Of course, it's just an exaggerated metaphor. There is a passage in the book about fear that I like very much.
The basis of fairness and justice is that people need to be responsible for something only after they have done it. After all, it is not a crime to want to do it but not to do it. The basic belief that society is related to personal responsibility is that people should be responsible for the actions they choose. If big data analysis is completely accurate, then our future will be accurately predicted. In the future, we will not only lose the right to choose, but also act according to the prediction. If accurate prediction comes true, we will lose our free will and the right to choose freely. Since we have no choice, we don't need to take responsibility. Isn't that ironic?
Here, by the way, another description of free will in the book.
In the field of philosophy, the debate about the existence of causality has been going on for centuries. After all, if everything has a cause and effect, then we have no freedom to decide anything. If every decision or idea we make is the result of something else. And this result is caused by other reasons. In this cycle, there is no human free will. ? All life trajectories are only controlled by causality. Therefore, philosophers argue endlessly about the role of causality in the world, and sometimes they think it is the opposite of free will.
The book gives an example, citing a film "Minority Report". When I saw this, oh, actually, I have seen this movie. I'm still a little excited to think about it. You can have a look if you are interested. Probably the police arrested the criminals in advance through prediction, but not through big data, but through a superhuman way. When everything you do is predictable, it means you are completely exposed to the sun. If it were you, would you be afraid?
Finally, two postscript are attached, one is a passage in the book, and the other is made up by myself.
Big data is not a cold world full of algorithms and machines, and the role of human beings still cannot be completely replaced. What big data provides us is not the final answer, but the reference answer. Help is temporary, and better methods and answers are still in the near future.
Big data will eventually affect us, and it will be a double-edged sword like other technologies. Use it well, be tempted, abuse, and be afraid. Just like nuclear technology, if it is used, it will benefit the earth. If it is abused, it will still explode if it gives you a diamond earth. I believe that the future development of big data will be a revolution in life, work and thinking, as the author said.
Reflections on the era of big data (2)
Last year? Cloud computing? Is it fried this year? Big data? Another surprise attack. It seems that overnight, all manufacturers have changed their flags and pushed up? Big data? Here it comes. As a result, CIOs of various enterprises have also turned their attention to the Heat? Big data? Here it comes. There is a cartoon from Weibo, The Programmer, which is very vivid. I think this picture really reflects the status quo of cloud computing and big data for SMEs.
But then again, The Age of Big Data is a good book.
Of course, many IT celebrities also strongly recommended it and wrote many comments to express their love for this book. Before reading this book, I was basically confused about the concept of so-called big data. Although I have paid attention to BI, which is also very popular now, I feel similar. It may be more data, more detailed data analysis and data mining. After reading this book, I feel that the previous idea can only be considered as a small half-massive data, while the other one: focusing on data relevance, not data accuracy, may be the biggest difference between big data and current BI, not just methods, but more ways of thinking. But frankly, it really takes time to test whether the data is more relevant or accurate. At least from the current data analysis methods, it is more inclined to the accuracy of the data. After reading this book, I have some questions in my heart:
1. What is big data?
I checked Baidu Encyclopedia, and it is defined as follows: bigdata, or huge amount of data, refers to the information that involves so much data that it can't be captured, managed, processed and sorted into more positive purposes by current mainstream software tools to help enterprises make business decisions within a reasonable time. The 4V characteristics of big data: quantity, speed, diversity and fidelity-this seems to be the definition of IBM.
Personal opinion: Mass data and mass storage are the basic prototypes of big data.
2. What kind of enterprise is big data suitable for?
It is true that the premise of big data is massive data. With massive data resources, we can find out the relevance of data and let it go.
Professional treatment, let it produce value for the enterprise. For telecom operators, large enterprises that use such massive user data on the Internet also have unique conditions on the road of applying big data, but what about small and medium-sized enterprises? Sales order data? If it is not a century-old shop, the estimated data is pitiful. 5. Only consumer data can be used. It seems that most manufacturers, for example, analyze consumers' buying behavior the most. Similarly, in the public sector, the role of big data may also play a very good role. On the contrary, I feel that most small and medium-sized enterprises seem to have a big problem in applying big data. The book says: Big data is the competitiveness of enterprises. It is true that data is the core intangible resource of an enterprise (if used well), but is it really appropriate for all data, or in other words: all enterprises have the competitiveness of big data? Will it appear that SMEs are making a mountain out of a molehill?
3. The impact of big data
When wave after wave of IT technology upsurge comes to our pavement, you have to start to meet the impact it brings to you before you are even ready. With the Internet of Things and cloud computing, big data began to appear. But what does it bring us?
1) Predicting the future Starting with the case that Google successfully predicted the possible future influenza, it shows that the application of big data can be used as a beacon for our lives. The essence is simple, technology changes the world.
2) Transforming the business opportunities brought by commercial big data will also give birth to a series of business opportunities and business models related to big data, and the potential value of data will continue to play a role. It is easy to imagine that there will be a data industry chain in the future, with specialized data collection, data analysis and data generation. Of course, IT companies have the greatest influence.
3) The Book of Easy Thinking says: Because there is a huge amount of data as the basis, we may pay more attention to the relevance of data rather than the fineness in the future. I still have reservations about this article.
Reflections on the era of big data (3)
Nowadays, when it comes to new media and the Internet, you have to mention big data. It seems that if you don't say this, you will be out. What's more, there are a lot of people who follow suit, and many empty talkers haven't even read the classic works on this subject seriously? Schon Berger's era of big data. Victor? Meyer? Who is Schoenberg? He is currently a professor of governance and supervision at the Internet Institute of the Network College of Oxford University, and was the head of the information supervision research project at Kennedy College of Harvard University. His consulting clients include Microsoft, Hewlett-Packard and IBM, and he is the real maker and participant behind the official Internet policy of the European Union. He has also served as a think tank for senior governments in many countries. This is known as: the prophet of the era of big data? Professor Oxford is awesome! So, the master said the golden rule? Not necessarily. You must do some homework when reading the works of the masters. If you can do your homework and have a corresponding theoretical basis, you can have an ideological dialogue with them.
Schoenberg discusses big data in three parts, namely thinking change, business change and management change. In the first part? Thinking change in the era of big data? In this paper, Schoenberg clearly shows his three viewpoints: first, many: not random samples, but all data; Second, more miscellaneous: not accurate, but miscellaneous; Third, better: not causality, but correlation. I disagree with the first point. On the one hand, it is very difficult to process all the data in terms of technology and equipment. On the other hand, is it necessary for everyone? Is it necessary to collect all the data for data analysis to judge simple facts? I have discussed this issue with Professor Jonathan Zhu from the City University of Hong Kong. Professor Zhu is an expert in communication research methods and data analysis. He thinks that we can find a method of mathematical statistics for analysis, and we don't necessarily need all the data. In connection with the relevance mentioned in Schoenberg's second point of view, I understand that the total data he said refers not to quantity but to range, that is, the random sample of big data is not limited to the target data, but also includes all data outside the target. I don't think big data analysis can rule out random sampling, but the method and scope of sampling should be expanded.
I agree with Schoenberg's second point. I think it is a good supplement to his first point of view, and it is also a reflection on precision communication and precision marketing. ? The simple algorithm of big data is more effective than the complex algorithm of small data. ? More macro vision and oriental philosophical thinking. I can't fully agree with Schoenberg's third point. ? Not causality, but correlation. ? Don't need to know? Why? Just need to know? What is this? . Communication is data, and data is relationship. In the era of small data, people only care about causality, but they don't know enough about correlation. In the era of big data, relevance plays an important role, which cannot be overemphasized, but it should not be completely ruled out. Where does big data come from? What is it used for? If we completely ignore causality and don't know the cause and effect of big data, it will also dispel the humanistic value of big data. Nowadays, in order to elaborate and spread their views, many scholars often make surprising remarks and completely deny old ideas.
The complexity and diversity of anything in the world are not simple either-or. Is Schoenberg also such a naive thinking of binary opposition? In fact, when reading, readers must see clearly what context they are saying, and don't fall into the misunderstanding of taking things out of context because of the shallowness of reading. For example, Schoenberg proposed? Not causality, but correlation. ? When he made this assertion, he also said in the book:? In most cases, once we have completed the correlation analysis of big data, we are no longer satisfied with just knowing? What is this? At that time, we will continue to study the causal relationship and find out the reasons behind it. Why? . ? [i] It can be seen that all the data and related relationships he said are in a specific context and are options in data mining.
One of the driving forces of big data research is commercialization. In the second part, Schoenberg discusses the business changes in the era of big data. Schoenberg thinks digitalization means anything is possible? Quantify? Quantitative analysis of big data is a powerful answer? What is this? This question, but still can't be completely answered? Why? . So I think we can't rule out qualitative analysis and qualitative research. There is no doubt that data innovation can create value. When discussing the role positioning of big data, Schoenberg still put it in the commercial system of data application, but not in the whole social system, but he discussed this issue in the second part of Management Change in the Age of Big Data. In the risk society, the problem of information security has become increasingly prominent, and data dictatorship and privacy protection have become a pair of contradictions. How to get rid of the dilemma of big data? Schoenberg is in the last section? Control? I tried to answer it, and it was basically a cliche. I thought, maybe Kevin? Kelly's loss of control can help us answer this question? At least it can provide more thinking dimensions. As Schoenberg said in his conclusion:? Big data is not a cold world full of algorithms and machines, and the role of human beings still cannot be completely replaced. What big data provides us is not the final answer, but the reference answer. Help is temporary, and better methods and answers are still in the near future. ? Thank you Schoenberg. Let the big data discussion return from natural science to humanities and social sciences. It can be inferred that "big data era" is not the final answer, nor the standard answer, but the reference answer.
In addition, before reading this book, you must have some basic knowledge and concepts of data science, such as what is data? What is big data? What's the difference between data analysis and data mining, and what's the difference between digitalization and dataization? Do some homework before reading, it will be easier to read.