Veri Bilimi

Yazarlar: Ahmet TOPRAK, Abdül Halim ZAİM, Feyzanur SAĞLAM TOPRAK

Konular:Mühendislik

Anahtar Kelimeler:Apache Storm,Big Data,Elasticsearch,Hadoop,Helmholtz Principle,Ibm Mq,Internet of Things,Rabbit Mq

Özet: Nowadays, IoT (Internet of Things) devices have reached quite high numbers. This situation brought with it the presence of high density, unstructured data. The difficulties in obtaining, processing, storing and visualizing this data made it necessary to use the components of the big data system. High density data should be taken from IoT devices instantly, meaningful data should be obtained from these unstructured data, the meaningful data should be stored and presented at the request of the user when needed. In this article, a model is designed to process the data obtained from IoT devices and transmit them instantly to the end user. In the study, unstructured data collected primarily from IoT devices were subjected to data pre-processing steps. Significant words were determined from the data obtained after the data pre-processing steps. For this purpose, the Helmholtz Principle has been applied. After meaningful word detection, it is directed to both Rabbit MQ messaging queue and IBM MQ message queue separately to instantly process data on the subject of each meaningful word. Apache Storm topology was used to instantly receive and process the messages transmitted to the queues. According to the results obtained, messages sent to the IBM MQ message queue are consumed 30% faster than the Rabbit MQ message queue.

---