MapReduce algorithm inspired by the map and reduces functions commonly used in functional programming. The use of this model is more beneficial when optimization of the distributed mappers in the MapReduce framework comes into the account. In standard mappers, each mapper operates independently and has no collaborative function or content relationship with other mappers. We propose a new technique to improve performance of the inter-processing tasks in MapReduce functions. In the proposed method, the mappers are connected and collaborated through a shared coordinator with a distributed metadata store called DMDS. In this new structure, a parallel and co-evolutionary genetic algorithm has been used to optimize and match the matrix processes simultaneously. The proposed method uses a genetic algorithm with a parallel and evolutionary executive structure in the mapping process of the mappers program to allocate resources, transfer and store data. The co-evolutionary MapReduce mappers can simplify and optimize relational data processing in the large clusters. MapReduce using a co-evolutionary mapper, provide successful convergence and better performance. Our experimental evaluation shows that collaborative techniques improves performance especially in the big size computations, and dramatically improves processing time across the MapReduce process. Even though the execution time in MapReduce varies with data volume, in the proposed method the overhead processing in low volume data is considerable where in high volume data shows more competitive advantage. In fact, with increasing the data volume, advantage of the proposed method becomes more considerable.

One of the important aspect of cloud computing is processing of amount of big data. MapReduce has been widely used as a powerful data processing model. It has efficiently solved a wide range of large-scale computing problems. MapReduce is a vital programming model for large-scale data processing in the cloud computing, which simplifies the development of traditional distributed program and provides a simple parallel programming method. On the other hand, Task scheduling is critical, which is an NP-completeness problem, plays a critical key role in cloud computing systems. In this paper, we propose a parallel genetic based algorithm to schedule the task on heterogeneous cloud environments. We prompt the algorithm on heterogeneous systems, where resources are of computational and communication heterogeneity. During the implementation of our method, we use Hadoop platform as the backend MapReduce engine. At the last part, through a series of simulation experiments, we prove that our approach has a much better runtime performance than other approach. The main goal of the proposed method is to use MapReduce framework to reduce the overall execution time of the program. The results of tests on a series of directional dag with random input indicate that the proposed method compare with three other existing method in this proposed method the speed of convergence is improved.

Clustering is one of the important methods in data analysis. For big data, clustering is difficult due to the volume of data and the complexity of clustering algorithms. Therefore, methods that can handle a large amount of data clustering at the reasonable time are required. MapReduce is a powerful programming model that allows parallel algorithms to run in distributed computing environments. In this study, an improved artificial bee colony algorithm based on a MapReduce clustering model (MR-CWABC) is proposed. The weighted average without greedy selection of the results improves the local and global search of ABC. The improved algorithm is implemented in accordance with the MapReduce model on the Hadoop framework to allocate optimal samples to the clusters such that the compression and separation of the clusters are preserved. The proposed method is compared with some well-known bio-inspired algorithms such as particle swarm optimization (PSO), artificial bee colony (ABC) and gravitational search algorithm (GSA) implemented based on the MapReduce model on the Hadoop framework. The results showed that MR-CWABC is well-suited for big data, while maintaining clustering quality. The MR-CWABC demonstrates an improvement of 7.13%, 7.71% and 6.77% based on the average F-measure compared to MR-CABC, MR-CPSO, and MR-CGSA, respectively.

The objective of this study is to verify the importance of the capabilities of cloud computing services in managing and analyzing big data in business organizations because the rapid development in the use of information technology in general and network technology in particular, has led to the trend of many organizations to make their applications available for use via electronic platforms hosted by various Companies on their servers or so-called cloud computing that have become an excellent opportunity to provide services efficiently and at low cost, but managing big data presents a definite challenge in the cloud space beginning with the processes of extracting, processing data, storing data and analyze it. Through this study, we dealt with the concept of cloud computing and its capabilities in business organizations. We also interpreted the notion of big data and its distinct characteristics and sources. Finally, the relationship between cloud computing with big data was also explained (extraction, storage, analysis).

Hadoop MapReduce framework is an important distributed processing model for large-scale data intensive applications. The current Hadoop and the existing Hadoop distributed file system’s rack-aware data placement strategy in MapReduce in the homogeneous Hadoop cluster assume that each node in a cluster has the same computing capacity and a same workload is assigned to each node. Default Hadoop doesn’t consider load state of each node in distribution input data blocks, which may cause inappropriate overhead and reduce Hadoop performance, but in practice, such data placement policy can noticeably reduce MapReduce performance and may increase extra energy dissipation in heterogeneous environments. This paper proposes a resource aware adaptive dynamic data placement algorithm (ADDP).With ADDP algorithm, we can resolve the unbalanced node workload problem based on node load status. The proposed method can dynamically adapt and balance data stored on each node based on node load status in a heterogeneous Hadoop cluster. Experimental results show that data transfer overhead decreases in comparison with DDP and traditional Hadoop algorithms. Moreover, the proposed method can decrease the execution time and improve the system’s throughput by increasing resource utilization.