Big data development engineers, first of all, you have to be familiar with relational databases, such as Oracle or MySQL, which is conducive to the development of data warehouses; I am familiar with Hadoop again, which is the most used technology in the field of big data. Its HDFS can realize distributed storage, and Yarn is an excellent resource scheduling framework.
2. Skills needed for big data
Skills that must be mastered:
Advanced Java (virtual machine, concurrency), basic Linux operation, Hadoop(HDFS+MapReduce+Yarn), HBase(JavaAPI operation +Phoenix), Hive (basic operation and principle understanding of HQL), Kafka, Storm/JStorm, Scala, Python, Spark (core +Spark SQL+Spark stream). )