#Visual studio code logo 640x480 manual
#Visual studio code logo 640x480 software
#Visual studio code logo 640x480 professional
#Visual studio code logo 640x480 windows

Developed Spark jobs using Scala in test environment for faster data processing and used Spark SQl for.

Data Frames are created by reading the validated Parquet Files and run the SQL queries using SQLContext to get the common transaction data from all the systems.

The validated parquet files of two or more systems got combined in curation module to get the common transactions data.

Storing the parquet data into hive data base with daily date partitions for further queries.

Create parquet files for valid records and invalid records separately for all systems.

Implemented Repartition, Caching and broadcast concepts on RDD’s, DF’s and variables to achieve better performance on cluster.

Files types Delimited, Position Based and Binary files are loaded in to SparkContext and validated against xml.

Prepared xmls for each source system like ATM, Loans, Teller etc to validate each record from HDFS source file and these xmls are validated by XSD.

Developed Sqoop job to pull PRDS (party reference data) to the HDFS location from Teradata.

#Visual studio code logo 640x480 professional

PROFESSIONAL EXPERIENCE Confidential, Charlotte Version Control: SVN, GIT, Clearcase, Visual Source Safe

Special tools: Maven, Autosys, GDB, Wireshark, Make.

#Visual studio code logo 640x480 windows

Operating Systems: Linux (CentOS and SUSE), HP-Itanium and Windows Programming Languages: Scala, Java, C/C++

#Visual studio code logo 640x480 software

Have flair to adapt to new software applications and products, self-starter, have excellent communication skills and good understanding of business work flow.īig Data Technologies: Apache Spark, HDFS, Yarn, Hive, MapReduce, Pig,Sqoop, Flume, Oozie, Kafka.Excellent written and verbal communication skills and team player. Experienced in Quality Assurance activities including peer reviews, casual analysis for defects and creation of checklists for improving deliverable quality.

#Visual studio code logo 640x480 manual

Created multiple tools that automate repetitive manual tasks and helps reduce effort by up to 50%.Involved in Process improvement activities that reduce operational costs.Possess superior design and debugging capabilities, innovative problem solving and excellent analytical Skills.Experienced in Agile methodology as a Subject Matter Expert and Technical Coordinator for work effort estimation, allocation and Technical Docs.Experience in working with relational databases Postgres and SQL programming.Experience on version control tools like Rational Clearcase, GIT, Visual Source Safe and SVN.Experience on IDE’s namely Eclipse, IntelliJ and Visual Studio.Technical expertise in GCC compiler, GDB, Wireshark, TCP/UDP Sockets technologies.Technical expertise in EJB, JBOSS, Restful Web services, Maven, JUnit and Arquillian Integration Framework technologies.Knowledge of NoSQL databases such as HBase, and DynamoDB.Good Knowledge of UNIX and shell scripting.Good understanding on Spark core, Spark SQL and Spark Streaming.Good understanding of HDFS Designs, Daemons, federation and HDFS high availability (HA).Experience in developing customized UDF’s in java to extend Hive and Pig Latin functionality.Strong experience in configuring and using Apache Hadoop ecosystem components like Hadoop MapReduce, HDFS, HBase, Oozie, Hive, Sqoop, Pig, Kafka, SPARK and Flume.Strong experience in Big Data and Hadoop Ecosystem tools like MR, PIG, HIVE, SQOOP, OOZIE, FLUME, HBASE, Kafka and SPARK.Strong experience in Big data & projects in multiple domains, tools in all phases of SDLC: Requirements gathering, System Design, Development, Enhancement, Maintenance, Testing, Deployment, Production support, System.Over 10 years of IT experience in leading multinational Information Technology firm with in - depth knowledge of Banking, Telecom, multimedia, PKI (Public Key Infrastructure) domains and Big data technology with multiple programming languages like Scala, Java and C++.