THE TREND OF THE FUTURE IS BIG DATA

Information technologies are evolving so rapidly today that it's difficult to stay informed, let alone follow them. Because by the time we become aware of a new technology, a better version or a technology that could replace it may have been released.

The concept of Big Data, which is now being discussed by many IT experts, has emerged. While it hasn't yet become widespread, it's already a goal for many and will undoubtedly be used very effectively in the future. So, what is Big Data?

In pre-internet information technologies, data didn't take up much space, and the amount of data needed to be stored was small. When the internet became a part of everyone's life, developing technology made smartphones indispensable, and data storage devices became more affordable, the amount of data circulating online increased dramatically. The development of social platforms and the widespread use of smart sensors that collect data in many areas of life have led to an incredible amount of data generation. For example, 5 billion searches are made on Google every day, 500 million tweets are sent daily, and 4 billion videos are watched monthly on YouTube alone. A passenger plane records 250 GB of data during a 45-minute flight.

2000 yılında dünyadaki veri miktarı 800.000 Petabyte (1 Petabyte= 1000Terabyte) iken 2020 yılında ise bu miktarın 35 Zetabyte (1 Zetabyte= 1000 Eksabyte ve 1 Eksabyte= 1000 Petabyte) olacağı öngörülüyor. 2012 yılında ise 2.7 Zetabyte veri olduğu tahmin ediliyor. Üstelik bu verinin %90’nı sadece son 2 yılda oluşturulmuş. Örneğin; Twitter her gün ortalama 7 TB, Facebook 10TB veri saklıyor. Firmalar bu muazzam verileri saklayabilmek için özel soğutma sistemlerine sahip sabit disk tarlası diyebileceğimiz yerler oluşturuyor. Aslında Büyük Veri kavramı her gün artan veri miktarından ziyade bu kadar çok veriden bir sonuç veya anlam çıkarmaya çalıştığımızda oluşacak yorumlamanın çok büyük olmasını belirtir.

In the past, every institution would have its own database and generate a report based on queries. But with the advancement of time, these reports no longer provide satisfactory results. The internet is currently overflowing with an incredible amount of data, and those who know how to analyze it can produce results, not just reports. Just like a face can be formed when thousands of images are placed in a specific sequence. But with Big Data, we can't identify patterns. We look at an incredible mass of data and try to see a picture, a pattern. Sometimes, that pattern isn't what we want. Those who don't know how to look, or who don't know what they're looking for, never see that pattern.

Let's give an example. Let's assume you have a server that extracts the raw data you want from this vast dataset using data mining technology. You own this server. If you were to run an SQL query, it would be so slow that it would be outdated and would produce nothing more than a report. For example, a report might say, "There are so many people in this city and this age range." But Big Data results don't just provide a report; they provide direct results and meaning. You can find direct conclusions like, "People's desires have shifted in this direction," "There's demand for this resort this summer." But if we can't do this with an SQL query, what kind of technology is needed?

Aslında uzun zaman öncesine kadar bu verilerin %80’i IT uzmanları ve kurumlar tarafından gereksiz bir yığın olarak görülüyordu. Fakat Google klasik yöntemleri bir kenara bırakarak ihtiyacı olan teknolojiyi geliştirdi ve bu verilerin ne kadar işe yarar olduğunu ispatladı. Google milyarlarca web sayfasının indeksini Google File System and keeps it as a database Big Table To process this data, MapReduce This technology involves thousands of low-cost computers working together in clusters. Amazon also uses a similar system.

To better understand Big Data, let's examine the components that make up it;

There are five components in the formation of a big data platform. These are variety, velocity, volume, verification, and value. Since they are generally described as the 5Vs, their English equivalents can be included.

• Variety: % 80% of the data generated is unstructured, and each new technology can generate data in different formats. It's necessary to deal with a variety of "Data Types" coming from phones, tablets, and integrated circuits. Furthermore, considering that this data may be in different languages and non-Unicode, it also requires integration and interconversion.
• Velocity: The rate at which Big Data is generated is very high and increasing. Faster data generation results in an equally rapid increase in the number and variety of processes requiring that data.
• Data Volume: According to IDC statistics, the amount of data reached by 2020 will be 44 times that of 2009. Considering the capacities and "big systems" currently in use, which we call "large," we need to imagine how they will cope with data 44 times larger! It's crucial to develop how the organization's data archiving, processing, integration, and storage technologies will cope with this volume of data, and quickly.
• Verification: Within this density of information, ensuring data flow is "secure" is another key component. It needs to be monitored at the appropriate security level, without being overlooked, and it needs to remain visible or private to the right people.
• Value: The most important component is value creation. Big Data, as defined by all the efforts mentioned above, should create added value for the organization beyond the data production and processing layers. It should instantly impact your decision-making processes and be readily available to help you make the right decisions. For example, a government agency making strategic decisions on healthcare should be able to see real-time distributions of diseases, medications, and doctors by region, province, district, etc. The Air Force should be able to see the current location and status of all aircraft in its aircraft inventory and track their retroactive maintenance history. A bank should be able to track not only the demographic information of a loan recipient but also their eating and vacation habits and, if necessary, their social media presence.

Source

These components constitute the raw data. Data mining yields the desired data. Unnecessary data is then purged. Analysis and modeling are then performed, and interpretation is derived based on the results. This is the operating system of the algorithm that processes this data. Because creating this system would be very costly, many institutions will prefer companies that provide this service in the future. While not currently in Türkiye, there are institutions abroad that have begun offering these services. Some mine data from platforms like Facebook and Twitter, while others mine data from websites, blogs, e-commerce sites, and surveys.

Big Data can be particularly beneficial for certain departments. For example, for marketing departments, knowing what products their customers want and what service or product they're interested in is crucial information. They can't learn this through standard reporting systems. Therefore, Big Data results are particularly suitable. If we were to list the most effective results that can emerge from Big Data:

• Understanding users' behavior and reactions online,
• Understanding trends and popular topics through social media analysis,
• Correctly target and optimize advertising campaigns,

We can list them as follows.

Even now, Big Data is being used effectively in many areas. For example;

• –Hospitals store individual data in their own digital environments in order to provide effective, individual, personalized medical services to their patients.

• Governments must work with the vast amount of data generated when processing and storing information and services for their citizens. For example, according to RTÜK (Radio and Television Supreme Council) decisions, television channels in our country are required to store the past year's broadcasts. The information to be stored is what we define as "Big Data."

• The acceleration of data production by producers and consumers on the Internet places the burden, primarily on service provider companies, to collate the growing amount of information, transform it into a meaningful form, and make it available for use.

• Banks have become content-rich, fast and convenient 24/7 branches that recognize their users, understand why they access the internet branch that day, and optimize the homepage and menu accordingly, send reminders to their customers, and offer customizable interfaces through the information they collect and store about their customers.

• Energy companies must store and process data about their subscribers' individual usage by using smart grids and meters.
• In the pharmaceutical industry, large genomic databases created for "cancer research", for example, must be constantly accessible to researchers.

Considering that it's already being used effectively in many areas, it's clear that it will become an indispensable technology in the future, as it will be used as a leverage over competitors in market strategies. In fact, future marketing strategies could be based entirely on this technology. Current estimates indicate that 2015 is the year for its full-fledged adoption.

Erhan Kılıç