A Review of Real World Big Data Processing Structure: Problems and Solutions

α
Khalid Imtiaz
Khalid Imtiaz
σ
M. Junaid Arshad
M. Junaid Arshad

Send Message

To: Author

A Review of Real World Big Data Processing Structure: Problems and Solutions

Article Fingerprint

ReserarchID

CSTSDE70F7C

A Review of Real World Big Data Processing Structure: Problems and Solutions Banner

AI TAKEAWAY

Connecting with the Eternal Ground
  • English
  • Afrikaans
  • Albanian
  • Amharic
  • Arabic
  • Armenian
  • Azerbaijani
  • Basque
  • Belarusian
  • Bengali
  • Bosnian
  • Bulgarian
  • Catalan
  • Cebuano
  • Chichewa
  • Chinese (Simplified)
  • Chinese (Traditional)
  • Corsican
  • Croatian
  • Czech
  • Danish
  • Dutch
  • Esperanto
  • Estonian
  • Filipino
  • Finnish
  • French
  • Frisian
  • Galician
  • Georgian
  • German
  • Greek
  • Gujarati
  • Haitian Creole
  • Hausa
  • Hawaiian
  • Hebrew
  • Hindi
  • Hmong
  • Hungarian
  • Icelandic
  • Igbo
  • Indonesian
  • Irish
  • Italian
  • Japanese
  • Javanese
  • Kannada
  • Kazakh
  • Khmer
  • Korean
  • Kurdish (Kurmanji)
  • Kyrgyz
  • Lao
  • Latin
  • Latvian
  • Lithuanian
  • Luxembourgish
  • Macedonian
  • Malagasy
  • Malay
  • Malayalam
  • Maltese
  • Maori
  • Marathi
  • Mongolian
  • Myanmar (Burmese)
  • Nepali
  • Norwegian
  • Pashto
  • Persian
  • Polish
  • Portuguese
  • Punjabi
  • Romanian
  • Russian
  • Samoan
  • Scots Gaelic
  • Serbian
  • Sesotho
  • Shona
  • Sindhi
  • Sinhala
  • Slovak
  • Slovenian
  • Somali
  • Spanish
  • Sundanese
  • Swahili
  • Swedish
  • Tajik
  • Tamil
  • Telugu
  • Thai
  • Turkish
  • Ukrainian
  • Urdu
  • Uzbek
  • Vietnamese
  • Welsh
  • Xhosa
  • Yiddish
  • Yoruba
  • Zulu

Abstract

Information sort and sum in human culture is developing in astonishing pace which is brought about by rising new administrations as distributed computing, web of things and area-based administrations, the time of enormous information has arrived. As information, has been principal asset, how to oversee and use enormous information better has pulled in much consideration. Particularly, with the advancement of web of things, how to handling huge sum continuous information has turned into an extraordinary test in research and applications. As of late, distributed computing innovation has pulled in much consideration with elite, yet how to utilize distributed computing innovation for substantial scale ongoing information preparing has not been contemplated. This paper concentrated the difficulties of huge information firstly and finishes up every one of these difficulties into six issues. Keeping in mind the end goal to enhance the execution of constant handling of substantial information, this paper manufactures a sort of real-time big data processing (RTDP) design considering the distributed computing innovation and after that proposed the four layers of the engineering, and various leveled figuring model.

References

53 Cites in Article
  1. Zhigao Zheng,Ping Wang,Jing Liu (2015). Unknown Title.
  2. O Tene,J Polonetsky (2012). Big data for all: Privacy and user control in the age of analytics.
  3. Lohrs (2012). Theage ofbigdata.
  4. R Bryant,R Katz,E Lazowska Big-Data Computing: Creating revolutionary breakthroughs in commerce, science, and society.
  5. R (2008). Unknown Title.
  6. Jonathan Overpeck,Gerald Meehl,Sandrine Bony,David Easterling (2011). Climate Data Challenges in the 21st Century.
  7. Alexandros Labrinidis,H Jagadish (2012). Challenges and opportunities with big data.
  8. Wang Wang Shan,Qin Hui-Ju,Xiong-Pai,Xuan Zhou (2011). Architecting Big Data: Challenges, Studies and Forecasts [J].
  9. Lu Weixing,Shou Yinbiao,Lianjun Shi (1996). Unknown Title.
  10. P Palermo (2004). The August 14, 2003 blackout and Its importance to China.
  11. Li Cuiping,Wang Minfeng (2013). Excerpts from the Translation of Challenges and Opportunities with Big Data.
  12. W Dobbie,R Fryer (2011). Getting beneath the veil of effective schools: Evidence from New YorkCity.
  13. H Jagadish,Johannes Gehrke,Alexandros Labrinidis,Yannis Papakonstantinou,Jignesh Patel,Raghu Ramakrishnan,Cyrus Shahabi (2014). Bigdata and its technical challenges.
  14. M Flood,H Jagadish,A Kyle (2011). Using Data for Systemic Financial Risk Management.
  15. Y Genovese,S Prentice (2011). Pattern-based strategy: getting value from big data.
  16. Albert-Lszl Barabsi (2012). The network takeover.
  17. Alexandros Labrinidis,H Jagadish (2012). Challenges and opportunities with big data.
  18. S Lohr (2012). How big data became so big.
  19. A Gattiker,F H Gebara,H Hofstee,J Hayes,A Hylick (2013). Big Data text-oriented benchmark creation for Hadoop.
  20. Min Chen,Shiwen Mao,Yunhao Liu (2014). Big Data: A Survey.
  21. Li Guojie,Cheng Xueqi (2012). Research Status and Scientific Thinking of Big Data.
  22. S Yadagiri,Prashanth Thalluri (2011). Information Technology on Surge: Information Literacy on Demand.
  23. Jeffrey Cohen,Brian Dolan,Mark Dunlap,Joseph Hellerstein,Caleb Welton (2009). MAD skills.
  24. Randal Bryant,Joan Digney (2007). Data-Intensive Supercomputing: The case for DISC.
  25. John Boyle (2008). Biology must develop its own big-data systems.
  26. Wang Yuan-Zhuo,Jin Xiao-Long,Chen Xue-Qi (2013). Network Big Data: Present and Future [J].
  27. Zou Guowei,Cheng Jianbo (2013). The Application of Internet of Things Technology in Spatio-Temporal Big Data Aggregation System of Smart City.
  28. Wang Qin Xiong-Pai,D Hui-Ju,Wang Xiao-Yong,Shan (2012). Big Data Analysis-Competition and Symbiosis of RDBMS and MapReduce.
  29. Tan Xiongpai,Wang Huiju,Li Furong (2013). New Landscape of Data Management Technologies [J].
  30. Chen Hai-Ming,Li,Xie Kai-Bin (2013). A Comparative Study on Architectures and Implementation Methodologies of Internet of Things [J].
  31. E A Lee,S Seshia (2011). Introduction to embedded systems: A cyber-physical systems approach.
  32. Ashish Thusoo,Joydeep Sarma,Namit Jain,Zheng Shao,Prasad Chakka,Ning Zhang,Suresh Antony,Hao Liu,Raghotham Murthy (2010). Hive - a petabyte scale data warehouse using Hadoop.
  33. Azza Abouzied,Kamil Bajda-Pawlikowski,Jiewen Huang,Daniel Abadi,Avi Silberschatz (2010). HadoopDB in action.
  34. Chen Songting (2010). Cheetah: A high performance, custom data warehouse on top of MapReduce.
  35. Rakesh Agrawal,Ramakrishnan Srikant (2000). Privacy-preserving data mining.
  36. Cynthia Dwork (2006). Differential Privacy.
  37. D Norman (2002). The Design of Everyday Things.
  38. Christopher Olston,Benjamin Reed,Utkarsh Srivastava,Ravi Kumar,Andrew Tomkins (2008). Pig latin.
  39. Rob Pike,Sean Dorward,Robert Griesemer,Sean Quinlan (2005). Interpreting the Data: Parallel Analysis with Sawzall.
  40. Ronnie Chaiken,Bob Jenkins,Per-Åke Larson,Bill Ramsey,Darren Shakib,Simon Weaver,Jingren Zhou (2008). SCOPE.
  41. Michael Isard,Yuan Yu (2009). Distributed data-parallel computing using a high-level programming language.
  42. L Fegaras,C Li,U Gupta (2011). XML query optimization in MapReduce.
  43. K Morton,M Balazinska,D Grosstman (2010). Para Timer: A progress indicator for MapReduce DAGs.
  44. Kristi Morton,Abram Friesen,Magdalena Balazinska,Dan Grossman (2010). Estimating the progress of MapReduce pipelines.
  45. Shi Huang Dachuan,Ibrabim Xuanhua,Shadi (2010). MR-scope: A real-time tracing tool for MapReduce.
  46. Meng Xiaofeng,Ci Xiang (2013). Big Data Management: Concepts, Techniques and Challenges [J.
  47. Y Chen (2012). We dont know enough to make a big data benchmark suite-an academia-industry view.
  48. Chen Yanpei,A Ganapathi,R Griffith (2011). The case for evaluating MapReduce performance using workload suites.
  49. Line Van Den Berg,Jérôme Euzenat (2012). Class ? En classe : jouer avec des classifications pour combiner mathématiques et informatique.
  50. Jiaqi Tan,Soila Kavulya,Rajeev Gandhi,Priya Narasimhan (2012). Light-weight black-box failure detection for distributed systems.
  51. Jun-Ming Zhao,Wen-Shuan Wang,Xian Liu,You-Fu Chen (2014). Big Data Benchmark - Big DS.
  52. S Patil,M Polte,K Ren (2011). YCSB++: Benchmarking and performance debugging advance features in scalable table stores [C.
  53. Yanpei Chen,Sara Alspaugh,Randy Katz (2012). Interactive Query Processing in Big Data Systems: A Cross Industry Study of MapReduce Workloads.

Funding

No external funding was declared for this work.

Conflict of Interest

The authors declare no conflict of interest.

Ethical Approval

No ethics committee approval was required for this article type.

Data Availability

Not applicable for this article.

How to Cite This Article

Khalid Imtiaz. 2018. \u201cA Review of Real World Big Data Processing Structure: Problems and Solutions\u201d. Global Journal of Computer Science and Technology - C: Software & Data Engineering GJCST-C Volume 18 (GJCST Volume 18 Issue C3): .

Download Citation

Issue Cover
GJCST Volume 18 Issue C3
Pg. 25- 36
Journal Specifications

Crossref Journal DOI 10.17406/gjcst

Print ISSN 0975-4350

e-ISSN 0975-4172

Keywords
Classification
GJCST-C Classification: H.3.m
Version of record

v1.2

Issue date

July 27, 2018

Language
en
Experiance in AR

Explore published articles in an immersive Augmented Reality environment. Our platform converts research papers into interactive 3D books, allowing readers to view and interact with content using AR and VR compatible devices.

Read in 3D

Your published article is automatically converted into a realistic 3D book. Flip through pages and read research papers in a more engaging and interactive format.

Article Matrices
Total Views: 5966
Total Downloads: 1556
2026 Trends
Related Research

Published Article

Information sort and sum in human culture is developing in astonishing pace which is brought about by rising new administrations as distributed computing, web of things and area-based administrations, the time of enormous information has arrived. As information, has been principal asset, how to oversee and use enormous information better has pulled in much consideration. Particularly, with the advancement of web of things, how to handling huge sum continuous information has turned into an extraordinary test in research and applications. As of late, distributed computing innovation has pulled in much consideration with elite, yet how to utilize distributed computing innovation for substantial scale ongoing information preparing has not been contemplated. This paper concentrated the difficulties of huge information firstly and finishes up every one of these difficulties into six issues. Keeping in mind the end goal to enhance the execution of constant handling of substantial information, this paper manufactures a sort of real-time big data processing (RTDP) design considering the distributed computing innovation and after that proposed the four layers of the engineering, and various leveled figuring model.

Our website is actively being updated, and changes may occur frequently. Please clear your browser cache if needed. For feedback or error reporting, please email [email protected]

Request Access

Please fill out the form below to request access to this research paper. Your request will be reviewed by the editorial or author team.
X

Quote and Order Details

Contact Person

Invoice Address

Notes or Comments

This is the heading

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.

High-quality academic research articles on global topics and journals.

A Review of Real World Big Data Processing Structure: Problems and Solutions

Khalid Imtiaz
Khalid Imtiaz
M. Junaid Arshad
M. Junaid Arshad

Research Journals